{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "dfdeecfc-8db4-4f06-a686-3d82fbefbf95",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# python3 -m venv venvJupyterLab\n",
    "# . venvJupyterlab/bin/activate\n",
    "# pip config set global.index-url http://mirrors.cloud.tencent.com/pypi/simple\n",
    "# pip config set global.trusted-host mirrors.cloud.tencent.com\n",
    "# pip install --upgrade pip\n",
    "# pip install scikit-learn jupyterlab jupyterlab-language-pack-zh-CN matplotlib ipympl ipydrawio"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26218171-bf2e-45d8-86d3-5220f8c82b05",
   "metadata": {
    "tags": []
   },
   "source": [
    "# 深度学习基础"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c480555-0b51-4e82-be5c-05efa441575b",
   "metadata": {},
   "source": [
    "## 应用领域"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72e87e9b",
   "metadata": {},
   "source": [
    "- 深度学习是一个复杂的机器学习算法，在语音和图像识别方面取得的效果，远远超过先前相关技术。\n",
    "- 深度学习在搜索技术，数据挖掘，机器学习，机器翻译，自然语言处理，多媒体学习，语音，推荐和个性化技术，以及其他相关领域都取得了很多成果。\n",
    "- 深度学习使机器模仿视听和思考等人类的活动，解决了很多复杂的模式识别难题，使得人工智能相关技术取得了很大进步。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30bdfe85",
   "metadata": {},
   "source": [
    "- 博弈曾被认为是人类智能最后的堡垒。\n",
    "- 自使用时间差分强化学习玩双陆棋的TD-Gammon开始，算法和算力的发展催生了一系列在博弈上使用的新算法。\n",
    "- 与双陆棋不同，国际象棋有更复杂的状态空间和更多的可选动作。\n",
    "- “深蓝”用大量的并行、专用硬件和博弈树的高效搜索打败了加里·卡斯帕罗夫。\n",
    "- 围棋因其庞大的状态空间被认为是更难的游戏，AlphaGo在2016年用结合深度学习与蒙特卡罗树采样的方法达到了人类水准。\n",
    "- 对德州扑克游戏而言，除了巨大的状态空间之外，更大的挑战是博弈的信息并不完全可见，例如看不到对手的牌。\n",
    "- 而“冷扑大师”用高效的策略体系超越了人类玩家的表现。\n",
    "- 以上的例子都体现出了先进的算法是人工智能在博弈上的表现提升的重要原因。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e66d13cc",
   "metadata": {},
   "source": [
    "- 机器学习进步的另一个标志是自动驾驶汽车的发展。\n",
    "- 尽管距离完全的自主驾驶还有很长的路要走，但诸如Tesla、NVIDIA、 MobilEye和Waymo这样的公司发布的具有部分自主驾驶功能的产品展示出了这个领域巨大的进步。\n",
    "- 完全自主驾驶的难点在于它需要将感知、思考和规则整合在同一个系统中。\n",
    "- 目前，深度学习主要被应用在计算机视觉的部分，剩余的部分还是需要工程师们的大量调试。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f7a6350",
   "metadata": {},
   "source": [
    "- 机器人学、物流管理、计算生物学、粒子物理学和天文学近年来的发展也有一部分要归功于深度学习。\n",
    "- 可以看到，深度学习已经逐渐演变成一个工程师和科学家皆可使用的普适工具。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f3b2cba",
   "metadata": {},
   "source": [
    "## 线性回归"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "013cbaeb",
   "metadata": {},
   "source": [
    "- 回归（regression）是能为一个或多个自变量与因变量之间关系建模的一类方法。\n",
    "- 在自然科学和社会科学领域，回归经常用来表示输入和输出之间的关系。\n",
    "- 线性回归（linear regression）可以追溯到19世纪初，它在回归的各种标准工具中最简单而且最流行。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c247fd2",
   "metadata": {},
   "source": [
    "### 线性回归基于几个简单的假设：\n",
    "- 首先，假设自变量x和因变量y之间的关系是线性的，即y可以表示为x中元素的加权和，这里通常允许包含观测值的一些噪声；\n",
    "- 其次，我们假设任何噪声都比较正常，如噪声遵循正态分布。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac5b37e6",
   "metadata": {},
   "source": [
    "### 数学模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "068088f8",
   "metadata": {},
   "source": [
    "- 考虑单个样本，当我们的输入包含d个特征时，我们将预测结果用$\\hat{y}$表示为：\n",
    "\n",
    "$\\hat{y} = w_{1}x_{1} + ... + w_{d}x_{d} + b$\n",
    "\n",
    "- 将所有特征放到向量x∈Rd中，并将所有权重放到向量w∈Rd中，向量x对应于单个数据样本的特征，我们可以用点积形式来简洁地表达模型：\n",
    "\n",
    "$\\hat{y} = w^{T}x + b$\n",
    "\n",
    "- 考虑整个数据集，对于特征集合X，预测值$\\hat{y} \\in R_n$可以通过矩阵-向量乘法表示为：\n",
    "\n",
    "$\\hat{y} = Xw + b$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23f63f24-b8ec-46d2-bf21-ce8adc011db8",
   "metadata": {},
   "source": [
    "<img src=\"drawio/A01.drawio.png\">\n",
    "\n",
    "线性回归模型的一个单层网络神经"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a80a863e-b1f0-4b2c-89e6-27027179a5ee",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%matplotlib widget"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4119c152-c39a-42f5-816d-3689eb0eab9a",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e46392ebbcd640c0a3dec1d3bddb4a97",
       "version_major": 2,
       "version_minor": 0
      },
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB0UElEQVR4nO3deVxU1fsH8M9lVARlcCENGRSXTMzdStNINDUqDUMkNftqi2UraGarqS1q5TehTFtdWvQb4ZTZz9BSKLJMLTUXSkoURFJxYcCd4fz+uM3EwCx3mH3m8369eNHcOXPvmQG6j+ec5zmSEEKAiIiIiAJGkKc7QERERETuxQCQiIiIKMAwACQiIiIKMAwAiYiIiAIMA0AiIiKiAMMAkIiIiCjAMAAkIiIiCjAMAImIiIgCDANAIiIiogDDAJCIiIgowDAAJCIiIgowDACJiIiIAgwDQCIiIqIAwwCQiIiIKMAwACQiIiIKMAwAiYiIiAIMA0AiIiKiAMMAkIiIiCjAMAAkIiIiCjAMAImIiIgCDANAIiIiogDDAJCIiIgowDAAJCIiIgowDACJiIiIAgwDQCIiIqIAwwCQiIiIKMAwACQiIiIKMAwAiYiIiAIMA0AiIiKiAMMAkIiIiCjAMAAkIiIiCjAMAImIiIgCDANAIiIiogDDAJCIiIgowDAAJAoApaWlJo8vXLiAEydO2HxdeXk5fvjhB4vP//777zhz5ozD/VPq22+/xeHDh51+3vPnzzv9nLUJIVxynaKiIpw7d87p5yUi/8YAkMjPZWdno2/fvqiqqjIeW7FiBa677joAwJ133olVq1aZfe2OHTsQFxdn8tovv/wSOp0OABAbG4ucnByL1/7rr7/w+++/2/VlzYQJE/Dtt9/i4sWLqKysNPt14cIFk9d8+OGHuHTpEgBAkiT8+eefxs/A8L5iY2Oh1WqtXvuWW25BXl4eAODMmTM4fPiw1a+anxkA/PLLLwgJCakTBBYWFuLJJ580Pv7tt9/wxhtvWO2LwQcffIB27dohOTkZQghFrzl37hx69OiBbdu21Xnuvvvuw/PPP2/19adOnbL53g1f5eXlivpERB4giMivDRw4UMydO9fk2EcffSRatWolhBBiy5YtolmzZuLjjz+u89qcnBwBQFy6dEkIIcShQ4cEAPHnn38KIYQAINauXWvx2lFRUQKAXV+1r69SqYyPW7duLZYtWyYefvhhi6+fOHGiyTkmTZoknnnmGWN/CwoKxNtvvy2uvfZacenSJfHnn38KSZLEkSNHrH6O7dq1M77XN9980+b72L17t8nrt23bJgCIc+fOCSGEmDt3rpgyZYooLy8XzZo1Ez/88IMQQoj//e9/onfv3lb7IoQQmzZtEsHBwWLq1KmiRYsWxveoRL9+/cTMmTNNjlVXV4s2bdqIzz//3Opr77zzTsU/y9TUVMV9IiL3auCyyJKIPO6rr77C/v37sX79epPjYWFhxtGZfv364csvv8S2bdtw9OhRXHbZZQgKCkJaWhpiY2MBAK+++iqeeeYZbNu2DeHh4ejQoYPiPixbtgyTJk2y2e7bb7/FsGHDFJ1z2rRpGDt2bJ3jzzzzTJ1j6enpuPbaa5GWlgYA0Ol0eOmll5CTk4MGDRrgs88+Q5s2bfDdd9+ZvC4hIQHNmjWz2IeBAweanR6vrKxEWFiYzfdw5swZVFRUQK1W45FHHsHMmTOxadMmVFVVoVGjRlZfu27dOowePRoTJkzA66+/jlGjRmHYsGGoqqrCK6+8YvY1M2bMwJEjRwDIo3hLly7FgQMHAAA33HADOnfujL///hsajcZkJLZ58+Zo3bq1yblSU1ORnp5ufJycnIyYmBgsWLDAeMzcz4eIvAcDQCI/df78eUydOhXPP/88mjRpgnHjxmHKlCmoqKhAXl4eLly4gLvuugslJSUoKirC4cOH8fjjj6OwsBDt2rXD4sWLMW/ePADAe++9h7/++guXXXYZysvLERT07+qRkSNHGv+7SZMmqKysdPl769Chg9kgtEWLFiaP9+3bh8zMTCQnJ+Ott94CACxfvhwpKSn4+OOPMXLkSKxYsQLl5eWYMmUKqqqqcPbsWajVauTl5aFZs2a46aabsHfvXvz999+YNGkSBg8ejEGDBuHSpUv4+++/6/Th7NmzdY5JkmT875CQEDz++ONo3Lix8VhaWhoWLlyIbdu24cKFCwgODjb7voUQWLBgAZ555hncddddeOeddwDIAVxWVhbGjx+PP/74A++//z4iIiJMXhsbG4tWrVoBAHr16mXyXNu2bfHee++huroa11xzjclzDz/8MBYtWmRy7NKlSyY/Z71eX+dY7SlwIvIuXANI5KeeeuopXLx4EV27dsW8efOwbt06dOjQAY8//ji2bNkCAIiIiMCdd96JJUuWYNeuXTh79ixiYmJw8uRJXLp0CW3atAEAfP/997j11luRk5OD5557Dvn5+cjPzwcALFmyxPj4119/tdif+fPnQ5KkOl+zZ8922Weg1+uN6wKzs7MBABUVFZAkCZWVlfj666/x+++/Y9++fTh9+jSWL1+Ozp074/Tp0+jevbvxc1y0aBFatGiBKVOmIDU1FQCwdetWREZG1vnq2LFjnX7k5+cjKysLALBr1y6TNX8A0LJlS2zbtg3XXHMNzp07h5CQkDrnKCoqwsiRI/Hkk09i1qxZWLp0KVQqlfH5kSNHYvPmzdi5cye6d++OFStWoLq62vj83Xffjfvvvx/Z2dkYP348goKCoNPpMH36dHTp0gWZmZn45JNPIISAEAI9e/bEypUr6wR/ALB48WKEhYUZv7744gu88cYbJsdWr15t74+LiNyIASCRn/r1119RWlqKe++9F8888wxmzpyJ6Oho/PHHH9iwYQMAOani3nvvxbBhw3DllVciKCgIlZWV+Pvvv9GwYUPj1F9kZCTi4uKwfft2JCUloUuXLujSpQsAQKPRGB937tzZap9GjRplDBbz8/ORnJzs0s+ge/fuWLBgAXr27ImioiIAwMaNG9G7d2+89tpr+PrrrwHAODV64MABtG/f3uQcgwcPxqhRoxAaGor+/fvj+uuvByBPARuCJSEEWrZsiZycHOPjbt26Gc/RpUsXtGvXDgDQuXNnXHbZZXX6aphuLy8vrzOFnJmZidjYWBQUFCA5ORkzZ840G0z37NkTERERGD16NO69914MGjTI5DyzZ8/GuXPnEBkZiZtuugkLFy7E//3f/+Gpp55CVVUVfvvtNwBAdXU1CgoKLP48U1NTTd776NGj8fjjj5scu+OOOxT8hIjIUzgFTOSnvvjiC4SHh2PWrFlYvXo1UlNTceHCBRw6dAiHDh1CSEgIXn75ZTRq1AiHDh3CwYMHcfToUcyaNQv9+/eHRqMxmerdvn07NBpNnelDe4SHhxsDR8NjewghEBQUhL1795rNXM7PzzdmNwNy+Zvp06fjhx9+wKZNmxAbG4vXXnsN8+bNw/vvv4+wsDAMGzYMv/32G/r164edO3eia9euFq9/4sQJZGdn49KlS3WmgIUQOHnypPFYo0aN6kxJA0BJSYnZaWKDsrIytGzZ0uRYhw4dMGvWLKSmpuLMmTN44YUXLL4+NDQUbdu2xWOPPYaCggLj8W+++QZvv/02tmzZAkmScNVVV2HRokW466678OSTTyIyMhJbt24FAOzduxdVVVUWP4uMjAxkZGTUOf7f//7X5LFhtJSIvA8DQCI/1aJFC2zbtg2vv/46vv/+ezRs2BCvvvoqnnzySbRs2RJBQUE4cOAAhg0bhhtuuAGtWrVCVFQUevXqhY8//tg4YmVw8803Iz8/32Q9m7tVVVWhYcOGOHnypHEau6ZTp06ZPM7NzYVKpcKuXbvw119/IS0tDcOGDcPtt9+OdevW4dZbb8Vbb72FvLw8TJ48GT/++CNeffXVOufdu3cvKioqcPfdd2PUqFHo16+fcQq4ptGjRxv/u1+/fsY+6vV6/PLLLwDkEUBrpVYOHTqEPn36mBy7+uqrcfXVVwMAgoODzQaWtXXu3Nk4gldRUYHx48fj9ddfR9u2bQEAhw8fxtatW5GQkICzZ8/i/vvvR9++fVFeXo6vv/4affr0MTsV/eabb2L+/Pkmxx544AFER0fjueeeMzmuJBmGiDyDU8BEfurYsWMYPXo0nnnmGURHR+PHH3/E2LFjUVFRgbKyMqSkpKBPnz547bXXcN999+G9997DZ599hpCQEDRr1gzDhw+vc87bbrvNZMoRkNeeGR5PmDDBpe+psrISTZo0QVxcHL799ts6XwMGDDBpP3bsWCxduhRBQUGIj4/HsGHDoFarERQUhJEjR6Jhw4a44YYbsG7dOmzfvh0lJSW48cYb65yjW7duOHXqFBYsWACtVosTJ05gxIgRVqeADcHfH3/8gejoaEyZMgWAnJk9a9Ysk2u89dZbxva7d+82TgfXNmnSJLNTv7W/NBqNyevCwsKwdu1aHD16FDfeeCMqKirw9ttvY+fOnXj//fcxa9YsxMbGonXr1li5ciVWrFiBUaNGmZxDr9fjzz//xIkTJ3D+/HmTr+rqalRVVdU5fvz4cWPdRSLyLhwBJPJTb775JoqLizF37lzMnDkTYWFhyMnJMY4A9ejRA0uXLkVFRQXGjBmDiooK49TimDFjAMgjaDUtW7bMZOeP2NhYLFmyBPHx8QBsT+muWLECK1asMDlWOxiy5OTJk7h48aIxk1WJjIwMTJ061fj41ltvNf53fHw8cnJy0KdPH7Rp0wbjx4/H0KFD60y/XnPNNZg2bRpSUlKMI2pHjhxBZGQkysvLjbtw1JwCVqlUxnV+rVu3xqRJk3D11Vdj9OjRGDx4sMn509PT8dxzz2H9+vXYv38//vzzT5Np7NqGDx9udvrV4PPPPzdmPNfUv39/dOvWDWvWrMGoUaOwd+9evPbaawgNDTW2efDBBzFnzhycOnUKd955p8nrjx8/jiuuuMLidQG5MLU5QmGRaiJyIzfWHCQiN9qxY4dYsWKF2Lx5s/j777/rPL9t2zahUqlE9+7dxdChQ0V5ebn48ssvRXl5ubFN7ULQtUFBIehly5YJIYQ4ceKEyM/Pr/N1/PhxIYQQ33zzjdVC0N9++62QJEmcPHnS4vUSExPrFII+c+aMaNOmjVi6dKnxWFxcnJg3b57x8cKFCwUAsX79eovnrlkIunv37uKVV14REydONFsAOSoqqs7raxeCfvbZZ8Vll10mmjRpInJzc4UQQjz++OOiQYMG4vfffzfbh4kTJ4rRo0db7KMQQixbtszs9Q2OHDkiWrVqJZo3b17n53r06FEhSZIYO3as1WsY/PXXX6Jbt26iQYMGolGjRiIuLk4cO3ZM0WuJyLM4BUzkp3r16oX//Oc/GDBgAFq3bo2jR48a18hVV1fjp59+gl6vR0xMDNatWwe1Wo2HH37Y6t6/jmjRooUxW7jmV+16dTVdd911OHToEAA5E7Zv375o3ry5XdfNy8vDqVOnkJ2djZ07d2L+/Pk4cuQIHnnkEQDyvsgrV64EAItb4tVUUlKC3bt3o1+/fgCAJ5980mQq+JtvvlHUr3379uHMmTNYu3YtBg0ahN27d+ONN96ARqPBPffcY1LCxZm2bduGkydP4sKFC3WSSebMmYMGDRrgyy+/xK5duyyeo7q6GosXL0afPn1w66234rbbbsOUKVPQuXNn9OzZkyVgiHwAp4CJ/FB1dTXee+897Nmzx/hVVlaGDz/8EM2aNcPs2bNRXFyMoUOHoqioCJIk4ejRozh8+LAxsHGW0tJSm3v8AkBxcXGdY8HBwYiKisKWLVuwdOlSfPDBB7h48aJxB4vaKisr6+zecdNNN+HgwYOYOXMmevfuDUAubN20aVNUVVVhwoQJKCsrM+6u0b59e6tJGkuXLkWrVq0wcOBALFu2zOb7siQ+Ph6PPvooBg8ejEOHDmHkyJGYOHEipk+fjl69etWZvja4dOkSTp8+bfG81jKMv/32W4wbNw7z58/HFVdcgdtvvx2xsbEYN24cZsyYgRUrViAvLw+zZ8/GbbfdhpycHJOC2+fOncOnn36K+fPno7KyEv/73/+QkJCA5ORkNGzYEO+//z4++ugj3HfffXj11VfxxBNPYOTIkRYLWxORB3l4BJKIXKRHjx7illtuETNmzBArVqwQ27dvF6+99ppo1KiReOihh8SJEydEWVmZaN26tRgxYoRISUkR/fv3NzlHzSlgS9Odtb+++eYb4+sd3QtYCCG2b98uIiIiREJCgtDr9SI/P9/q62tPAf/888/i4YcfFmq1WowaNUqkpaWJ4OBgMXPmTHHbbbeJli1bir179wohhFi1apUICgoSU6ZMEVVVVUIIIQoLC8W2bdtEkyZNxKpVq0z23VUyBXzu3DmRn58vXn/9daFSqYznNdi6dauIjIwUiYmJ4uLFi0IIeZ/gkJAQUVBQYNJW6c+g9hRwdXW1ePHFF0VQUJCYPn268fjcuXPFkiVLxLhx40RoaKjIzs4WQghx6tQp0bVrV3HZZZeJn376SQghxPfffy/UarVo0aKFePnll8WZM2eM5xk9erR4/PHHjY9PnDghZsyYIdRqtVCr1aK4uLjOz5WIPIsBIFEAOXXqVJ2b8ZYtW0SnTp1EZGSk2Lx5s8lzNQPAI0eOmF3DV/urZmBQcw2gLebWAJ4+fVo0b95cXHfddeL06dM2z1F7DeDbb78tGjduLMaOHSu2bNliPH7o0CGxYMEC0aZNG2PwZ/Dpp5+K9u3bi8rKSiGEHEgHBQWJrl27iu3bt4vBgwcb+zJx4kQxbdo0UVFRYfxau3atSQBWUlIiAIiQkBDx8MMP1+lzfHy8eOihh0zW4126dElMnjy5znueOHGiGDFihCgtLbX4lZ6ebjYAHDJkiHj//ffrXP/zzz8Xl19+udi6davJ8RMnToiBAweKDz74QAghxPnz58WKFSuMaxhrqh0AGlRWVoo1a9bUOU5EnicJwfQsIvJeubm5uOaaa9CkSRO7X3v+/HlcvHgRarXa7POnTp0yu6awoqJCUQ07wzq9mgWz7WVt79/6EEKgurraZJs4QO6rpX7qdDqzn9GFCxfQqFEjj9Z+JCLXYABIREREFGCYBUxEREQUYBgAEhEREQUYBoBEREREAYYBIBEREVGAYSFoB1RXV+PIkSMICwtjlhwREZGPEEKgoqICbdq0cSiL35cxAHTAkSNHEB0d7eluEBERUT0UFxdDo9F4uhsewQDQAYY6YcXFxRbrjBEREZF30el0iI6OVlTv018xAHSAYdpXrVYzACQiIvIxgbx8KzAnvomIiIgCGANAIiIiogDDAJCIiIgowHANoIsJIVBVVQW9Xu/prhD5JJVKhQYNGgT0Wh0iImdjAOhCFy9eRGlpKc6ePevprhD5tNDQUERGRqJRo0ae7goRkV9gAOgi1dXVKCwshEqlQps2bdCoUSOOYBDZSQiBixcv4vjx4ygsLMQVV1wRsEVbiYiciQGgi1y8eBHV1dWIjo5GaGiop7tD5LNCQkLQsGFDHDp0CBcvXkTjxo093SUiIp/Hf0q7GEcriBzHvyMiIufiCCAREZEf0lfrkVeUh9KKUkSGRSKubRxUQSpPd4u8BP9ZTUaTJk2CJEkWvw4ePFiv844YMQKpqamK2h46dAgtWrTAli1b6nWt+srNzUWzZs0Utd25cyfefvtt13aIiMgB2nwtYjJiMHjFYIzXjsfgFYMRkxEDbb7W010jL8EAkIxeffVVFBQUoKCgAO+++y6ioqKMjwsKCuq9YfaqVavwyiuvKGrbrl077N+/H/3796/XtdyBASAReTNtvhbJmck4rDtscrxEV4LkzGQGgQSAU8BeT68H8vKA0lIgMhKIiwNULhrBb9WqFVq1agUA+P3339GgQQN06tTJ4fPau9l2RESEw9ckIgpE+mo9UrNTISDqPCcgIEFCWnYaEq9M5HRwgOMIoBfTaoGYGGDwYGD8ePl7TIx83FNmz56NUaNGYd26dejVqxfS0tIAACtXrkTXrl0REhKCq6++Gps3bza+Jjk5GbNnzwbw71TrN998g6uuugrh4eEYP348KioqAACVlZUm082TJk3CY489hqeffhotWrRA27Zt8f7775v0adGiRYiOjkZ4eDjS0tLw7LPPYtSoUVbfx3fffYc+ffogNDQUQ4cOxYEDB0ye37BhA66++mqEhISga9euWLNmjfH933333di1axckSUJMTAwA4M8//8TIkSPRpEkTaDQavPjii/X4dImIHJNXlFdn5K8mAYFiXTHyivLc2CvyRgwAvZRWCyQnA4dr/R2XlMjHPRkE7ty5E6+++irmzp2Lp59+GlVVVdi4cSOef/55/Pzzzxg+fDhuv/12iwWwz549i1deeQXvvPMOVq5ciU2bNuGll16yeL0PP/wQKpUKGzduxKRJk/DAAw9g7969AOTA84knnsDzzz+PvLw8SJKEN954w2r/Dxw4gJtvvhmDBg3Cli1bcOedd2L69OkmbbKzszFlyhT8/PPPeOCBBzBu3DgUFxfjsccewyuvvILY2FgUFBTgu+++AwDk5eWhb9+++P7777F48WK88cYb+Oyzz+z5WImIHFZaUerUduS/OAXshfR6IDUVEHVH8CEEIElAWhqQmOi66WBrLl26hHXr1pnUN/zggw+M/92jRw8sWrQI+/btw9VXX2329e+//75x9OzJJ5/EypUrLV6vV69exgCxd+/eWLNmDXJzc3HVVVchIyMDTzzxBCZPngwAWLhwIX7++Wer/X/nnXdw9dVXY+HChcb+FhcX4/XXXze2qfnfPXr0wPvvv4+ff/4ZycnJaNWqFRo1amQyPX733Xcb/7tv374YO3YsvvvuO4wZM8ZqX4iInCkyLNKp7ch/cQTQC+Xl1R35q0kIoLhYbucJffv2rVPc+qOPPsJ1112H1q1bIyQkBBUVFaisrDT7+qZNmxqDPwDo1KkTysrKLF6ve/fuJo9rtt+zZw+GDx9u8nztx7Upec26deswZMgQtGnTBiEhIdi7d6/F9wMA+/btwx133IH27dujSZMmWLJkidX2RESuENc2Dhq1BhLM7zwlQUK0OhpxbePc3DPyNgwAvVCpwpF5pe2crXZR3rVr1+LBBx/E2LFjsXbtWuzYsQNqtdri61W1hi0lSYIwN9ypoL25AsGXLl2yq/+17dixA6NHj0Z8fDxWr16NX3/9FV26dLHY/vz58xg2bBgaNmyI999/H1u3bsWECROsXoOIyBVUQSpkJGQAQJ0g0PA4PSHdaxNA9HogNxdYtUr+rtd7ukf+iwGgF4pUODKvtJ2rffPNNxgzZgxSU1Nx7bXXIjg4GDqdzi3X7tu3L9avX29ybNOmTQ69JicnB/3798fzzz+P6667DlFRUThcY0hWpVJBX+P/Svv27cOxY8ewfPly3HjjjbjqqqtQWFjoyNsiIqq3pNgkZKVkIUodZXJco9YgKyULSbFJHuqZdd6Y+OjPuAbQC8XFARqNnPBhbmBMkuTn47xkBF+j0WDJkiXYvHkzzp07h1mzZiE4ONgt13788ccxZswYREZGYsCAAVixYgV2796N1q1bW3zN5MmT8frrr+ORRx7B5MmTsWvXLpPEEY1Gg507d2LDhg1o0qQJZs2aZTIK2aFDB+zfvx/r1q1Dp06dcPnll0MIgXfeeQfXX3893nnnHezbtw/t27d36XsnIrIkKTYJiVcm+sxOIIbEx9r3PEPiY1YWkOSdcavP4gigF1KpgAx5BB9SrWUchsfp6Z5JADHn4YcfRt++fXHTTTfhsccew5w5c9C0aVO3XHvkyJFYtGgR5s+fj+uvvx4VFRV49NFHIdX+4GqIiopCdnY2fvzxR/Tv3x8ffPAB0tPTjc8nJycjJSUFY8aMwYQJE3DvvfeiY8eOxucHDhyIe+65B3fccQdmz56NNm3a4K233sJLL72E+Ph4hIeH46677nLl2yYiskkVpEJ8TDzGdR+H+Jh4rw3+bCU+AnLiI6eDnUsS1hZfkVU6nQ7h4eEoLy+vs+bt/PnzKCwsRPv27dG4ceN6nV+rlf8oaiaEREfLwR//JWTZ1KlTodPpTDKTybc54++JiLxTbq483WtLTg4QH++ca1q7fwcKTgF7saQkudSLu3YC8UXHjx9HUVER+vbtazz2008/4fbbb/dgr4iISClvT3z0VwwAvZxK5bx/8fijJUuWYNGiRVi4cCG6du2KlStXYs+ePfjyyy893TUiIlLA1xIf/QUDQPJpzz33HBo3boznnnsOJSUluOqqq/Dll18a9zQmIiLv5muJj/6CSSDk04KCgjBjxgwUFhbi4sWL2LFjB4YMGeLpbhERkUK+lvjoLxgAEhERkUclJcmlXqJMSxdCo2EJGFfhFDARERF5HBMf3YsBIBEREXkFJj66D6eAiYiIiAIMA0AiIiKiAMMAkIiIiCjAMAAko0mTJkGSJItfBw8edOj8y5cvx5YtW+x+3cGDByFJEk6fPq2o7fz58+vROyIiosDBAJCMXn31VRQUFKCgoADvvvsuoqKijI8LCgqg0WgcOn99A0B7MAAkIiKyjVnAXk5frUdeUR5KK0oRGRaJuLZxUAW5Jie+VatWxh00fv/9dzRo0ACdOnVyybWIiIjIczgC6MW0+VrEZMRg8IrBGK8dj8ErBiMmIwbafK1H+7Vq1SpceeWVCAkJQa9evbB27Vrjc9999x369OmDkJAQdOnSBf/73/8AADExMfjuu+8wdepUSJKE2bNnWzz/7t27ccMNNyA0NBTXXnstfv31V5Pnt2/fjvj4eISEhKBDhw547733AMgjjIMHD0Z5eblx2hoAjh49igkTJiA8PBytW7fGY489hkuXLjn5UyEiIvIdHAH0Utp8LZIzkyFgujFiia4EyZnJyErJQlKs+0ujb9u2DZMnT0Z6ejquvfZa5ObmYsyYMdi6dSu6deuG0aNH4/7778eHH36IH374AZMnT0bPnj3x3XffISUlBcOGDcOkSZPQokULs+c/ffo0hg0bhiFDhiAjIwOFhYVITU01afPNN98gMTER6enp2LNnD+6//3707NkTo0ePRsOGDfHggw+aBI07duxAREQENmzYgMrKSkyZMgXR0dF44oknXPpZEREReSsGgF5IX61HanZqneAPAAQEJEhIy05D4pWJLpsOtiQ9PR1PPPEE7rvvPgBAjx498OOPPyIrKwsajQYnTpzA2LFj0a1bN3Tr1g1JSUnGaeWQkBBERERYnVZetWoVQkND8eGHH6JBgwbo3bs3KisrMXHiRGObp59+2vjfvXr1wqefforvv/8e1157LaKiohAUFGRyjYSEBCQkJBgfT5kyBTk5OQwAiYgoYHEK2AvlFeXhsO6wxecFBIp1xcgrynNjr2T5+fl4+eWX0bhxY+NXVlYWjh07hhYtWmDixIkYPHgwUlNTsWnTJlx22WV2nX/Pnj0YMmQIGjT4998mw4cPN2nz008/YcSIEWjbti1CQ0Oxbt06VFZWWjzn4cOHce+996JTp05o2rQpnnrqKavtiYiI/B1HAL1QaUWpU9s5kxACL7zwAkaNGmVy3DClu3z5cmzevBmff/45Jk2ahHbt2mHDhg0ICQlRdP6gIOv/Jjly5Ahuuukm3HPPPXjyySfRsmXLOlPEtSUmJqJVq1Z48803odFo8L///Q+bN29W1B8iIiJ/xBFALxQZFunUds7UtWtXHDhwAF26dDF+NWnSxDjNW1hYiIEDB2LBggXYv38/CgsLsX79egCASqWCXq+3ev6+ffti06ZNqKqqMh7btGmT8b+3bNmCli1bIj09HXFxcbjyyitx4MAB4/O1r3Hy5En8+uuvePvtt3HzzTeje/fuOHzY8ugqERFRIGAA6IXi2sZBo9ZAgmT2eQkSotXRiGsb5+aeAdOnT8eKFSswb9487Nq1CytWrED37t3x/fffo6ioCF27dsWLL76IPXv2YN26dTh58iRiYmIAAB06dIBWq8XOnTtx/Phxs+cfM2YMqqqqMH78ePzyyy9Ys2YNZs6caXxeo9GgpKQEmZmZ2L59O+644w7odDrj8zExMThz5gxWrFiBvXv3Qq1WIywsDO+++y52796N2bNn4+uvv3bpZ0REROTtGAB6IVWQChkJGQBQJwg0PE5PSHd7AggA9O7dG5mZmVi1ahWuvfZazJkzB6+99hpuuOEGtG3bFmvXrsWaNWtwzTXXYNq0aViwYAF69eoFQE7euHDhAq6//nr8/PPPZs/fpEkTbNiwAX///Teuv/56zJkzB4sWLTI+f+211+KZZ57Bgw8+iBEjRuD666/HoEGDjM9HR0fjhRdeQFpaGu6//340aNAAH330EVauXIkBAwagpKQEM2bMcOlnRERE5O0kIUTdVFNSRKfTITw8HOXl5VCr1SbPnT9/HoWFhWjfvj0aN25cr/Nr87VIzU41SQiJVkcjPSHdIyVgiDzFGX9PREQG1u7fgYJJIF4sKTYJiVcmum0nECIiIgoMDAC9nCpIhfiYeE93g4iIApReD+TlAaWlQGQkEBcHqDgO4fMYABIREZFZWi2QmgrULJ6g0QAZGUASVyL5NCaBEBER+TG9HsjNBVatkr/bqMZlpNUCycmmwR8AlJTIx7We3ZaeHMQAkIiIyE9ptUBMDDB4MDB+vPw9JsZ28KbXyyN/5tJEDcfS0pQHk+R9GAASERH5IUdG8PLy6r6uJiGA4mK5HfkmBoBERER+xtERvFKFO40qbUfehwEgERGRn3F0BC9S4U6jStuR92EASERE5GccHcGLi5OzfSXzO5JCkoDoaLkd+SYGgERERH7G0RE8lUou9QLUDQINj9PTWQ/QlzEAJBMxMTGQJAmSJCEkJARdunTB888/j4qKCqdeJzc3F82aNQMgb/PVunVrZGVlKXrt8uXLsWXLFoeuv3z5cuMexf7okUceQWJiosvOf/DgQUiShNOnTwMAXnvtNb/+PIl8jTNG8JKSgKwsICrK9LhGIx9nHUDfxkLQVMeSJUswdOhQnDt3Dnv27MELL7yAH3/8Ed9++61Lrte4cWPs27cPzZs3V9R++fLlGDVqFPr37++S/viD1157DXo31mdIS0vD5MmT3XY9IrLOMIKXnCwHezWTQewZwUtKAhITuROIP2IASHVcfvnl6NSpEwCge/fu6N27N2JjY1FcXIzo6GiXXLNly5YuOW+gCgkJcev1GjZsaBzRJSLvYBjBM7eTR3q68hE8lQqIj3dFD8mTOAXsLtXVwPHjnvuqrq531w039qCgIOPU6datWzFgwACMHj0aAFBYWIibb74ZoaGhiI6OxvTp03H+/HnjOb777jv06dMHoaGhGDp0KA4cOGByjYiICOTm5hoff/311+jbty9CQkLQu3dvfPPNNwDkKervvvsOU6dOhSRJmD17NgBgx44duP766xESEoIOHTpg3rx5qK7xnlevXo0uXbogNDQUY8aMQVlZmdX3bJiiLigowPDhw3HNNdcAAMrKynDHHXcgLCwMrVu3xj333GOcBgWAXbt2YeDAgQgNDUVcXByys7NNpkolSUJeXh7uv/9+NG/eHGVlZaiursazzz6Lyy+/HGq1GsOGDUN+fr7xnEuWLEFMTIzxnDt27AAAnD59GmPHjkXz5s3RsmVLPPzwwzh79iwAYPr06Zg0aZLxHCUlJRg1apSx39OnT8fFixcB/Dudm5OTg2uvvRZhYWG4+eabUWpHfYdFixYhvsYdQpIkfP755xg6dCiaNGmCgQMH4o8//jA+b+tzJCLnSEoCDh4EcnKAlSvl74WFnL4lAILqrby8XAAQ5eXldZ47d+6c2Ldvnzh37px84NgxIeRReM98HTum6D21a9dOfP7550IIIaqqqsSePXtEQkKC6N27txBCiGXLlolWrVqJfv36iaysLFFUVCT0er3o3r27uOuuu8Qvv/wi1q9fL2JjY8UTTzwhhBDir7/+EiEhISItLU3s2rVLLF26VDRv3lyEh4cbr9uyZUuRk5MjhBDi559/Fg0bNhQvvfSS2Llzp3jllVdEWFiYOHr0qDh48KC49tprxbPPPisKCgrEiRMnRGVlpWjdurV44oknxM6dO4VWqxWRkZHirbfeEkIIsXnzZtGgQQMxd+5csWvXLrFgwQIRGhoqevbsafFzyMnJEY0bNxb9+vUTK1asEAUFBUIIIW699VaRkJAgfvrpJ/H999+LAQMGiDFjxgghhDh16pRo1aqVGDdunPj1119FZmamiI6OFgDEqVOnhBBCABDdunUTL730ktixY4fQ6/XitddeE506dRJfffWV+OWXX8TkyZNFu3btxJkzZ8TOnTtFcHCwWLlypdi1a5dIS0sTbdu2FRcuXBBTp04V11xzjdiyZYvYtGmT6NOnj3jqqaeEEEI8/vjjYuLEiUIIIS5evCiuuuoqkZCQIH788Uexbt060alTJ/Hggw8KIYQoLCwUAES/fv3E+vXrRW5urujcubNISUmx+PkYXmN4X2+++aYYNGiQ8XkAokuXLmL16tViy5Yton///uLaa681Pm/tczSnzt8TEZEDrN2/AwUDQAf4awDYsGFDERwcLIKCgkRQUJAYMWKEKCwsFELIAWDjxo1FaWmp8TUbNmwQHTt2FHq93njs66+/FtHR0UIIIWbMmCHi4uJMrjNnzhyLAeC4cePEf/7zH5P2ZWVlxv8eNGiQWLhwofHxu+++K+Lj403aL1myxHjNlJQUcdddd5k8f/fdd9sMAAGInTt3Go/t379fNG3aVFRWVhqP5efnC5VKJS5cuCAWL14s2rdvLy5dumR8/p133qkTAKanp5tcq02bNiI3N9f4WK/Xi8jISLFx40axevVqERkZKc6fP298/ujRo0IIIUaOHCmmTp1qPH7q1Clx8eJFIYRpALhu3ToRHh5u8nv6ww8/iODgYKHT6YzBnOHzF0IIrVYrIiIiLH4+SgLAZcuWGR//+uuvAoCoqKiw+TmawwCQ/E1VlRA5OUKsXCl/r6rydI8CCwNAIbgGkOpYuHAhbrzxRgQHByMqKgqNGjUyef6KK67A5Zdfbnycn5+PwsJChIaGGo8JISD9s9J4z549GD58uMk5hg8fjtdff93s9X/77Tc88cQTJsesrRHMz89HXl4eGjdubDxWXV2NDh06GK//9NNP17n+r7/+avGcABAWFoaePXuaXOfMmTN1+qLX63Hy5Ens2bMHQ4YMQYMG//5Z1X7fABBXI+1Op9PhyJEjuOmmm0zaXLx4EceOHcMtt9yCiIgI9OjRA3feeSeSkpLQrVs3AMCUKVOQkpKCgoICJCUlYdSoUWjYsGGd6+3Zswe9evWCWq02HrvuuusghMAff/yBiIgIAPJ6T4NOnTrZnCa3pfb5AODEiRM2P8eav1tE/kirNb8uLyODU7PkPgwAqY6oqCh06dLF4vNBQaZLR4UQuPrqq7FixQqT46p/0sRqt1dCmNu/yErb2267DXPnzjU5bggI63N9c68TQqBNmzZms6FbtWpl9jqXLl2yel7D+1y3bh3atGlj0k6j0aBp06b45ZdfsHbtWqxduxYDBgzAfffdh9dffx233HIL9u/fj88++wwrVqzA448/ji+++AI33HBDvd6vqkZan2SpdoSD5xPyrIPVz5HInxn25639vzjD/rwsr0LuwiSQf0yaNAmSJGH58uWuuUDLlsCxY577cmGWbdeuXfHnn38aA8cuXbqgTZs2aN26NQCgb9++WL9+vclrNm3aZPF8vXr1qvP8iRMnjP+tUqlMSpx07doV+fn56Ny5s/H6LVq0MAZU9l7f2vs8evQoQkNDjdfp2LEjmjVrhqCgIPTt2xebNm1CVVWV4uuEh4ejTZs2+Pvvv43n7NKlC5o0aYKmTZuiuroaR44cQVJSEpYtW4bNmzdj4cKFOH36NI4dO4bw8HCkpqYiNzcXd999t9lR1auuugo7d+6ETqczHvvpp58AAJ07d7b7c3CUrc+RyFF6PZCbC6xaJX93Y0Ukqxzdn5fImfh/WwA//vgjMjMz64zAOFVQEHDZZZ77cuGNdejQoWjfvj2Sk5OxefNm/PDDD7j11lsxdepUAMDkyZOxa9cuPPLII9i1axc+/PBDvPHGGxbPl5aWhk8//RRz587Frl278Oqrr6J9+/Y4duwYAKBDhw7QarXYuXMnjh8/jjvvvBMVFRWYOHEitm3bhm+++QaDBw/GK6+8AgBITU3Fp59+ihdffBG7d+9Geno6MjMz7X6fV1xxBUaOHImkpCRs3LgR27ZtQ0pKCu68804AwJgxY1BVVYXx48fjl19+QWZmJl5++WWb533yyScxdepUrFq1Cr/99htmzpyJnj17QqfT4bPPPkOfPn3wySefYO/evfjiiy/QsmVLqNVqTJw4Ebfddht++OEHbN++HVu3bkX79u3N/nyioqJwxx13YMuWLcjOzsakSZNwzz33mEwL28Mwurdt2za7X2vrcyRyhFYLxMQAgwcD48fL32Ni5OOepNcDb77p2P68RE7lsdWHXqKqqkr06tVLzJo1S/Tt29dk4botdiWB+IiaWcDmLFu2zGzyRHFxsUhKShJNmzYVarVaTJw4UZw+fdr4/ObNm0Xv3r1F48aNxQ033CBWrVplMQlECDmJpE+fPqJx48aiR48e4uuvvzY+99dff4m+ffuKJk2aiLVr1wohhNi7d68YOnSoCAkJEREREWLq1KkmiRNr1qwRV155pQgJCREjR44Ub731ls0kkJr9Mzh9+rS4++67RfPmzUVoaKi47bbbxOHDh43P5+fni7i4ONG4cWPRt29f8cUXXwgAxs8CgNixY4fJOaurq8XLL78soqOjRXBwsLj66qtFXl6e8fk33nhDtG/fXjRu3Fj07t3bmDBSVlYmxo0bJ8LDw0WzZs3EHXfcYUzKqJkEYvj5JCYmiqZNm4rLLrtMTJs2zZhwUTuhQwghdu/eLaz976GqqkrEx8eL0NBQIYT5JJCa77OiokIAMCYT2foca/PVvydyr9WrhZCkujlwkiR/rV7tuX5pNMpz9lau9Ew/AwmTQISQhLBjsZUfeuutt7BgwQLk5+fj+uuvxyOPPGJSP80anU6H8PBwlJeX1xlJOX/+PAoLC9G+fXuT5AQKLLt27cI111yD8+fPc3rTAfx7Ilv0enmkz9IImyTJiRaFhe7dxcLSmj9rcnJYeNnVrN2/A0VA35HKysowc+ZMpKenK7qpXLhwATqdzuSLqKbs7GyTxz/99BM6duzI4I/IxfLyvG961dqaP3OU7M9L5CwBfVd66qmn0K9fPyQmJipqP2/ePISHhxu/XLUtGvmm/fv3Y9SoUUhLS8POnTvx6aef4vnnn8ejjz7q6a4R+T2lG9fYscGNw2wFpTUZ9usdPVp+HRNByNUCNgDcunUrPvnkE6vJCLU9/fTTKC8vN34VFxe7sIfkazp37oyNGzfil19+wXXXXYfU1FQ88MADmDJliqe7RuT3IiOd284Z7Ak2DZME6enek7hC/i1g6wC+++67OH/+PGJjY43H9Ho97r33Xnz00UfYuHFjndcEBwcjODjYnd0kHzNw4EDkMYWPyO3i4uQ1fiUl5qdcDWsA3Tm9ak+wWXvEz611AXNzgR9/BCIigLvuAkJCXHxB8gYBGwDOmTMHaWlpJsfGjh2LcePGYfz48Z7pFBER1YtKJe+kkZz873SqgaGueXq6exNAbAWlgNwfc9O9Qsj9TksDEhNd1O/jx4Haxdd375br1ZDfC9gp4KioKHTr1s3kq3HjxoiKijJbS42IiLxbUpI8YhYVZXpco/HMDhuGoBT4NwitzdpaP5cmrkyeXDf4A4B6FMkn3xSwI4DuEuBVdoicgn9HpFRSkjxilpcnr8GLjJRH4tw58le7P1lZdff+tYdTE1e++856jZla+5KT/2IAWMP27duddq6GDRsCAM6ePYsQrqcgcsjZs2cB/Pt3RWSNSuVddfQMQenLLwOzZtn/eqckrlRWykOh5eXW2736qhMuRr6AAaCLqFQqNGvWzLh9WWhoKCRLcwBEZJYQAmfPnsWxY8fQrFkz4xZ0RL7ovffsa++0xJVZs4AXXrDeZvVq98+Rk0cxAHShyy+/HACMQSAR1U+zZs2Mf09EvsiemoCAkxJXPv5Yzuq15uabga++cul+8eSdGAC6kCRJiIyMRKtWrXDp0iVPd4fIJzVs2JAjf+Tz7F3Hp9HIwV+9BuXOngWaNLHdrrBQLjhIAYkBoBuoVCrewIiIApjSdXzPPQfceKMDiSu9ewM7d1pvs2QJ4OUF6vV670nk8VcMAImIiFxMaaHq2bPrGejk5spbiFhzxRXA3r2AlydTabV1s6Y1GrmkDpcpOg8n/YmIiFzMWk1Ah9b76fXyCWwFf59/Duzf7xPBX3Jy3fWShp1RuD2e8zAAJCIicgOnF6qeMAFooGAiTwhg1Cg7T+5+er088mduhNRwLC3NevFsUo5TwERERG6SmAiEh8sztoBcrzA+3s6Rv/x8oGtX2+3+/hto3dr+TnqIrUzpmjujeFOdR1/FAJCIiAKKpxIMzK1tW77czrVtSurJvvACMHNmfbroUUozpZ26M0oA4xQwEREFDK1WrnwyeDAwfrz8PSbG9WvLHF7b9tJLyoI/IXwy+AOUZ0o7ZWcUgiS4yWa96XQ6hIeHo7y8HGq12tPdISIiKwxBWO27niGuqtc6PAX0ejnItDS9acgALiw0MxJ57Jiyadx9+4DYWEe76lGGz8lWprTZz8lOvH9zBJCIiAKAJxMM7FnbZkKSbAd/48fLJ/Dx4A9wYaY0mcUAkIiI/F69gzAnsHtt2yefKJvuraqS2/oRp2dKk0VMAiEiIr/nyQQDpWvWopqfBSQFW7jl5Ph1GmxSkpwtzZ1AXIsBIBER+T17EgycnSWsZBeQXQ36ovvNv1o/UffuwG+/1b8jPkSl8usY1ytwCpiIiPyeIQizNLMqSUB0NFBW5vwsYWtr2wbhO1QLCd0v2Qj+KisDJvgj92AASETkAXq9XAx41Sr5O3c3cC0lCQZjxwIpKa7Zhqz22rYg6CEgIRfx1l/44YfysGETBVPDRHZgAEhE5GaeqkUX6KwlGHz6qRyMuzJLOCkJOHgQKB0+EXolK7CEAO66q/4XJLKCdQAdwDpCRGQvd9Wi89RuF77A3GeTlycH4rY4lH/xxx9Aly622/nYFm6+iPdvJoEQEbmNrVp0kiSPMiUmOhasmdtyTKOxc8sxF/CWoNRcgoHLs4SVlHWZNQuYPbueFyCyD6eAiYjcxB216BzecsxFvH3a22XbkL38svIt3Bj8kRsxACQichNXjzJ5crcLa7w1KK1JaZZwXJzCEx47Jr/oueest9u71/wPjMjFGAASEbmJqze79+RuF5Z4a1Bam1O3IWvQwPYavrFj5Q+ga1d7u0rkFAwAiYjcxOmjTLV4crcLS7wxKLXE4W3IVq2Sf4i2otmqKrktkQcxCYSIyE0Mo0zJyXKcUHNUzBmb3bt6hLE+vDEotaZe25CdOweEhto++caNwJAhTusrkSM4AkhE5Eau3Oze1SOM9eGNQalTXXON7eCvWzc52mfwR16EI4BERG6WlASMGAEsXgwU/KWH1C4P/W4sRYvmkdBXx0EVVL8hQGsjjID8+L77nPAG7KBkH1yNxr1BqTWKS+jk5QE33GD7hJWV3MWDvBJHAImI3EyrBTp2BKa+q8XihjF468xg/OfL8Ri8YjBiMmKgza9/WqylEUaDWbPcW37FqckVLqYoW7m6Wu64reBv+XJu4UZejQEgEZEbGYOMMC2QkgyoTaONw7oSJGcmOxwEHjwIzJlj/nl3l19x5bS3syjJVq66625lkaoQwMSJzu0gkZNxKzgHcCsZIrKHXi+Pvh0u0QNpMXLwZ2a9ngQJGrUGhamF9Z4ONl7LQgauYeq1sNA1o2/6aj3yivJQWlGKyLBIxLWNA4TKK3YCMSc31/JWcFdgP/bjStsnOXLEhxczBhbev7kGkIjIbYwlUWLygHDLtVEEBIp1xcgrykN8TLxj17J0jRrlV+q9t60F2nwtUrNTcVj3bwc0ag0yEjKQFO8Fw31mWMpCFuYi9Nqef97ycCuRl+IUMBGRmxiDjKbKap6UVtS/Noqnyq9o87VIzkw2Cf4AoMQJU9uuVHvg7inMUxb8VVcz+COfxACQiMhNjEFGpbJpwsiw+k8neqL8ir5aj9TsVAjUXVlkOJaWnQZ9tYe3/TDDkK18GY5DQMI8PGP9Bbt3y8OoSvb5JfJCDACJiNzEEGSgKA4o1wDCfPAgQUK0OlpeN1dPZWW22zi7JmBeUV6dkb+aak5texuVCjj4dzCOoZX1hikpcuDXrZt7OkbkIgwAiYjcxFASRRIqIPuf2ih1gkD5cXpCukMJIFOn2m733/86NwlD6ZS1I1PbLvHpp4AkQVV10Xq7S5fktkR+gAEgEZEbGUqiaCqSgMwsQGdaGyVarUFWShaSYuufLGErAcTgssvqfQmzlE5ZOzK17VTnzslTuGPHWm2mX/+tPOrXgHmT5D/420xE5Gb/7jebhJIjiTgemofL2pciKlwul1LfkT8DTyWAxLWNg0atQYmuxOw6QEN5G0emtp2mXz9g61brbWJjgX374CWVaoicigEgEZEHqFSG8isqAPFOPben9t9VBamQkZCB5MxkSJBMgkDJCVPbTvHDD8oWPlZUAE2bur4/RB7CKWAiIhv0erlQ8KpV8ne99yWxmjAkm1hKUJUk5yeAGCTFJiErJQtRatOpbY0TprYdYtjCzdabXrZMnu5l8Ed+jjuBOICVxIn8n1YrbxFWc02dRiMnc3jDFmaWGLacA0y3NzMEha7egs3cTiAeG/m77z7ggw9st+PtMGDw/s0A0CH8BSLyb4Ygqvb/Jd0VRDnKXPAaHQ2kp3t3v52moADo3Nl2u5ISoE0b1/eHvAbv3wwAHcJfICL/5em9dJ1Fr4fX7r/rUkoKND/3HPDii67vC3kd3r+ZBEJEZJYn99J1pn+TTQLEK68ATz1lu51hTSBRgGIASERew5vWjXmqlArVU1mZssKGv/0GdO/u+v74EG/6uyP3YQBIRF5Bm69FanaqyVZiGrUGGQkZHskc9VQplUBW70AkNFQu6mxNUhKwerX16wfgdLm3/d2R+3ANoAO4hoDIObT5WiRnJtcpHmyoHefq8iHmbvyAvAawpMR8cqivrAH0FfUKRDIzgTvusH3yS5ds7uLhq9nejvD0350n8f7NANAh/AUicpy+Wo+YjBiTG39Nht0jClMLXTItZe3GD3i2lEqgsDsQOX8eCAmxfeING4Bhw2xf38ezvevD0393nsb7NwtBE5GH5RXlWbwJAYCAQLGuGHlFeU6/tuHGXzvZo6Tk38AvKwuIMq1pDI3GP4MCT9BX65GanWp26zjDsbTsNOir/6m+PWCA7eCvc2c5mlMQ/On18j8AzA2FGI6lpXl/8W97efLvjrwD1wASkUeVVijLolDaTilbN35Jkm/8hYWGfXsDa22YuygNRHZ9vgR9kh+1fUKdDggLU359P8n2tpen/u7IezAAJCKPigxTlkWhtJ1S9t74/enm701sBRhSNVD9AgDYCP4++AC45x77rx+g2d6e+rsj78EAkIg8Kq5tHDRqDUp0JWanAQ1rkeLaOnfj2kC98XsbawHGu18Ck39VcBIHlrIHara3p/7uyHtwDSAReZQqSIWMBDnjwrDo38DwOD0h3ekL0QP1xu9tDIFIzZ99pxOAmK0g+Dt82OH9e+Pi5DWdlmpCS5K8fV6cn8VBnvq7I+/BAJCIPC4pNglZKVmIUptmW2jUGkWlKPR6IDcXWLVK/q5kwX6g3vi9Te1ARMwGCt608aKnn5YDv9rZOfW5vurfjO/avwuGx+np/rnm09G/O/JtLAPjAKaREzlXfQoBO1K/zZAFDLDMi6cdGdoPbTZutd3QRVu4mfs9io6Wgz9//x0IxJ1AeP9mAOgQ/gIReZYz6rcF8o3fKxQXA23b2m63axfQo4dLuxKIO4EEKt6/GQA6hL9ARJ6j18s7dVjK5LVnpw5fv/H7bP+VjOTdfrscpRM5Ee/fzAImIh/lzPptKpXvlnnxyS3M5swBZs+23U7BFm5EVD/8yyIin8QyLpanwA07mXjdGsbKSmVFmpcsAaZMcX1/yITPjiRTvTAAJCKf5CtlXFx1U1W6k0liopfcxJUmbnBVkkf45EgyOYRlYIjIJ/lCGRetVl6nOHgwMH68/D0mxjlL2uyZAveo119XFvyVlTH48xBbe2JzCaZ/YgBIRD7J2+u3ufqm6vVT4Hq9/IN4/HHr7R56SA78WrZ0T7/IhK2RZEAeSVZSW5N8CwNAIvJZSUnyOrfa9YA1Gs+uf3PHTdWrp8AlSVnyhhDAW2+5vj9kkc+MJJPTBXwAKElSna+YmBhPd4uIFEpKAg4eBHJygJUr5e+FhZ5dt+SOm6qtKXAE6XHZNbkoab4KuQdzoa92wxDOp58qm+7Nz+d0r5fw+pFkcpmATwIpKCgw/ndRURGSkpLwxBNPeLBHRGQvbyvj4o6bqmEKfPQYPdAuD2haClRGAofigC5rgIRUHA8/jAlfyO01ag0yEjJct70Xkzx8klePJJNLBXwA2KlTJwDAsWPH8NBDD2HevHl48MEHPdwrIvJlbrupxmrR8sVUnLhUY7jxTEsg9ESdpiW6EiRnJjt/j1elgZ+LtnAjxxhGkktKzMfmhoLq3BPb/wT8FDAAlJeX4+abb8a0adMY/BGRw9yRoazN1yI5M9k0+AP+Df5qXVtAvrunZac5Zzr4t9+UBXQLFvxbl4a8jrcnU5HrMAAEMHPmTOzYsQMzZszATTfdhD/++MPTXSIiH+bqm6q+Wo/U7FRjUGd6AdQJ/gwEBIp1xcgrcnBFvyQBPXvabieE7SxgO+j1QG4usGqV/J2Zqc7hrclU5FoMAAFMmzYN69evx1dffYWGDRti+PDhOHfuXJ12Fy5cgE6nM/kiIjLHlTfVvKI8HNZZyTKxobSinosPe/dWNpJ35ozT1/q5sqYieWcyFblWwK8BBICYmBhj5m9mZiaio6Oxfv16jBo1yqTdvHnzMGfOHPd3kIh8UlKSvBOHs3cCqXcA94/IMDsXH548qaxO34gRwNq19euUFT635Z2P8rZkKnItBoC1hIaGomPHjvj777/rPPf0009j2rRpxsc6nQ7R0dHu7B4R+RhX3FTtDuCMJESrNYhra8fiQw9n9/rclndEPiKgp4DLyspw22234cSJfzPmzp07hwMHDqBdu3Z12gcHB0OtVpt8ERE5yt61bXFt46BRayBZXuyHOssDhdw2PSEdqiAFkdITTygL/g4cMInOnL1Oj4WKiVwjoAPAiIgInDlzBiNGjEBubi62bduGCRMmoGXLlrjxxhs93T0iCgD1WdumClIhI0HOMqkTBP4T6OGc6ZSt6owGnyUrKAFj2MJtwQLr7YKC5OirfXuH3ostLFRM5BoBHQACwJo1a9CnTx+kpKRgyJAhuHjxItavX49GjRp5umtE5Occ2S84KTYJWSlZiFLXyjLRaYDM1cBrR4HlOcDqlcDyHKzqX4jkq2wEf/Zs4VZraM9Vex+zUDGRa0hCsCx7fel0OoSHh6O8vJzTwURkF71eHh2zNL1pKMBbWGh9bZu+Wo+8ojyUVpSiYEck3n0uDiXF/74gOlouOWM1SeKzz4CUFNudXr8eGD7cZe/FHMO5bRUqrs+5KXDx/s0kECIip9HrlWf82rO2zVoSiSpIhfiYfxp0B569086sYyckeTjrvZhjqKmYnCx3tWY3WKiYqP4YABIROYFWK2er1gyENBo5eDE3+uaqtW2Ks46duIWbq9fpGWoqmvt8bY5uEpFZAb8GkIjIUfVZ/+axtW179igL/ubPV7yFmzveCwsVEzkX1wA6gGsIiKi+6988srbNRTX9uE6PfA3v3xwBJCJySH3r1Ll6v2AT11yjLPirrKxXQWe3vhcicgoGgEREDnBk/Zsr9wsGAJw6JUdg27dbb3fzzXLg16RJvS/l8vfiIs4uXE3kK5gEQkTkAEfXv7lqv2BPbOHmsvfiIvYm7hD5E64BdADXEBCR161/e/ZZYO5c2+3+/BPo2NH1/fFShsSd2j8zQ9zszaOW5DjevzkFTETkEK9Z/2Yo16Ik+BMioIM/vV4e+TMXsBuOpaVxOpj8GwNAIiIHeXz9myQpizCFcOqUr6+qb+IOkT/hGkAiIiewd/2bPbuGWLRuHXDrrbbbff01kJBg58n9l6sLVxP5AgaAREROonQXDqckH3ggycNfeKwIN5EX4RQwkR9jiQvvU59dQ0w0aqQs+KuuZvBnQVycHHBb+hglCYiOltsR+SsGgER+SquVs1MHDwbGj5e/x8QoCDDIZRxKPvjjDzkyuXTJ+kUyMhRv4RaovCZxh8iDGAAS+SGHR5nIJeqdfCBJQJcuti8gBPDYYw71MVB4PHGHyMO4BpDIz9gaZZIkeZQpMZEjHK5WO9GjpETZ64zJByNHAl99ZfsFlZWKd/FwSvKJn/C1wtVEzsQAkMjP2DPKpCRhgerHXKJHRISy12rUOkAKt91w3Dhg5UqH+hToO18oTdwh8jcMAIn8DEtceJ6lXSbKyqy/TpKAaiEBIxRcxM4ED0t9MiwL4LQnUWDhGkAiP8MSF55lbQq+ptrJB09hvhz82XLggN3Bnz/ufMEMdyLHcASQyM8YSlzY2pvWl0pc6Kv1yCvKQ2lFKSLDIhHXNg6qIO9cqGVrCt4gIgI4fhyQUI1q2H4vokULSCdOuKRPrl4W4OyfH6eyiRzHAJDIzxhKXCQny8FezSDQF0tcaPO1SM1OxWHdv3d7TZgGk6MzcEVVktct3Fc6tb5wIXDnBGWlWiQIaEKBDG39AhxPLgsw+/NTa5CRkIGkWPvfDKeyiZyDU8BEfshfSlxo87VIzkw2CR4A4LCuBLP2JmP8i1qvq2+oZGr9JmQrCv6GYCMkyJGOIyV8PLUswNLPr0RXguTMZGjz7Xsz/jiVTeQpkhAsFV9fOp0O4eHhKC8vh1qt9nR3iOpwVckPd5QS0VfrEZMRUyd4MBISoNMA6YWQ/plC9YbgVq+XA1JLU/ACykf96hz7Z/q+sNC+z9tWn+p7XqvXtPHzkyBBo9agMLVQ8XRwbq5c0NyWnBxm9pJ1vH9zBJDIrxlKXIwbJ393xs3dXTuM5BXlWQ7+AEASQHgx0C7Pq0Z/LO0ycRYhioI/CdVmgz/ASqHoevap5mNnLwuw9fMTECjWFSOvSPmbYYY7kfMwACQixdy5w0hphcK7eFO5XX2DI1eoOQXfEX9CQEIIzlt/0cKFWLVSAAqCxPoEOO5eFqD056f45wxmuBM5E5NAiEgRd+8wEhmm8C5eadrOW0Z/kpKApNEK9+P950ONzFXWvL4Bjjt3vlD681P8cwYwYIDcV2ujvCqV3I6IrOMIIBEpUu99bOsprm0cNGoNJEsjYkICyqOBQ6b1bLxi9Of22+vOtZpTUWESURtK+Fh6qSQB0dGOlfBxxbIAc2z9/CRIiFZHI66t8jfz44+2p/j1erkdEVnHAJCIFHH3+itVkAoZCfLCtTpBhKFgcnY6IOQIxhnBkcMqKuSOfPGF9XZjxsiBX9OmJofduVbP1YWUrf38DI/TE9LtqgfINYBEzsMAkIgU8cT6q6TYJGSlZCFKXWvhmk4DZGYB+fLCNa+obyhJgIJswiBJQDs20+Lz7lir565EHks/P41ag6yULLvrAHINIJHzsAyMA5hGToHEE6VEjNeusZNEwY5IvPtcHEqK/71IdLQc/HmkBMyCBcATT9hs1gF/oRAdFH9Oriq1Y6mQsiGIdkVCiLN2AvHk7yD5F96/GQA6hL9AFGgMwQNgfocRd9Xhc0cdQpuqqxVd9DTC0Ryn6xx3d606w5RvSgpw8qT5Nt4UQFn6GXvL7yD5Nt6/fWwKmLEqkWd5yw4j7kpksEiSFF1UgjAb/AHuXadmmPIdOtRy8Ad4Tykda1PU3vI7SOTrfKoMTJs2bZCSkoKxY8fiuuuu83R3iAKSO0uJeJ0NG4CbbrLZbCi+wUYMtdrGXevULE35WuPJJAqle/0G7O8gkZP41BTwqlWr8Pnnn+Prr79Gy5YtjcFgnz59PNIfDiETBRAlZV0A6KuE16xTM6yZs1a+xxxPbaVmq7/eNEVNvo33bx+bAh43bhwyMzNRVlaGjIwMHDt2DMOGDcOVV16JWbNmYd++fZ7uIhH5G7VaWfCn1wNCeGTbNUts1W6szd5SOs4uJePuWpNEgcynAkCD4OBgJCYmYvny5fjqq6/QtGlTvPjii+jevTt69OiBuXPn4sCBA57uJhH5sr/+kiOiigqrzapfXYDcHIFVnwYZgyBvWadmz1SuvcGpK0rJsM4fkfv41BpAg19++QWZmZnIysrCsWPHMGLECMyaNQuxsbHQarXIysrCzJkzoff0rvBEXsorsmgd5NL3oHC6V7taIDXVdNRKo5FHAL1hnZo96ww1GuWldJSu07MX6/wRuZHwIU8++aTo0KGDCA4OFiNGjBCffPKJOHPmjNm2hYWFLu9PeXm5ACDKy8tdfi0iZ1m9WgiNRgj59i1/aTTycV/hsveQnGx6UktfOp1YvVoISar7lCTJX97weVZVyZ+LuX4avlq0EOLbb+W29pzT0vkkSYjoaOXns6e/jpybqCbev4XwqSSQIUOGYNy4cUhOTkbz5s093R0uIiWf44kiwM7mivegL6+EqlmY7Ya33w5otT6VrODsunm5ufJ0ry31TSRhnT9yB96/fWwN4KZNmzB58mSvCP6IfI1eD6Smms9MNRxLS/t3Ib+r94qtD3vfgyKSpCz4E8K4wM2XkhWcvR7R1ev0vGX9JJG/88k1gERkP3uClpMnYXVtm6fY8x5sjj4tXAhMm2b7on/+CXTsaHLI15IVnLke0R3r9Lxh/SSRv2MASBQglAYja9bIgZ6zF/g7g1MCLyGAINuTH5Vogq7RlSiMAWrHHb6YrGDYPcVRcXHyPwZs1TlUWkrGEmf1l4jM86kpYCKqP6XByMcf12+K1R1Txg4HXpKkKPiTIBCGSovTuIYgyFKysL319HyJN9U5JKL6YwBIFCCUBC3NmgFlZZbPYWltmytqwplT78Dr228VlXa5CdmQYBr9mhtNVBoEAd63jtIZuE6PyPcxACQKENaCFkAO7k6fVnaumkGRIWuz9to8w5SxM4PAeo0+SRIwbJjNc0sQ2IC6+/xaGk20FQQB7gmKPSUpCTh4UM72XblS/l5YyOCPyFf4VBkYb8M0cvJFWm3dBA97GUp8eKocirn3EB1dq5BxixbAqVM2z6WCHtVm/i2stO/mClKvWeM95Xb8oeg3kbPx/s0A0CH8BSJfZQgKSkrkNX3Wpn1rqh0UubomnDUWA5sDB+pk7Zr1yivQdprh9Jpz3lQj0Fyg7A3Z3ESexvs3s4CJApIhwzI3177gDzCdYvVkORSzWaIKt3AzRHuJemD2bDkgOnny36ft2RatNqeWqnGAq7ZrIyL/wDWARAHMnsDM3AJ/rymHMnassuCvvNwYERkSV2bN+jf4a9ECmDPHsbVs3lAj0CUFs4nIrzAAJApgSgOzhQvNB0UeL4dy5ox8kU8/td7uttvkyOefqR5LiSunTskjgmvW1L9L9QmK9dV65B7Mxardq5B7MBf6asciM1/aqYSIPINTwEQBTGlR30cfNb9ezZCVm5wstzW3js5lNeHsnO41sDU6Jkny6FhiYv36bW+hZG2+FqnZqTis+zdi06g1yEjIQFJs/YYhvWEUkoi8G0cAiQKYM4r6ur0mXEaGsuDvjz/MRmCuHh2z5zPV5muRnJlsEvwBQImuBMmZydDm169mjNdMzROR12IASBTgnBHAuaUmXM3hOWuCg+W2nTubfdodo2NKPlN9tR6p2akQqBukGo6lZafVazrY41PzROT1OAVMREhKkqc8HakX59K9W+s53WuOu0bHbH2meUV5dUb+ahIQKNYVI68oD/Ex8WbbWCqF49GpeSLyCQwAiQiAiwO4+tq0CbjxRtvt1q0Dbr5Z0SntXaPnCGufaWmFsiFGS+1s1fgzjEKaa1PfEjdE5D8YABKRd3LiqF9N3jI6FhmmbIjRXDulNf6cMbJLRP6JO4E4gJXEiVygdWvg2DHb7fR6IKj+y5gVbSfnQvpqPWIyYlCiKzG7DlCCBI1ag8LUQqiC/o3YvGmnESJfxfs3k0CIyFuUlsrRi63gb+5ceejLgeAPcFPiihWqIBUyEuR0YQmmo52Gx+kJ6SbBH2B/FrNeL+/4smqV/J3Fn4kIYABIRN5AkoA2bWy3EwJ4+mmnXNLiXsJulBSbhKyULESpTdOFNWoNslKyzNYBtCeL2bDbyeDBwPjx8veYGPk4EQU2rgEkIs+ZPVvee82W06eB8HCnXdZWAoU7JcUmIfHKROQV5aG0ohSRYZGIaxtXZ+TPQGl2ckGB/PFyL2AiModrAB3ANQRE9XT+PBASYrvdLbcA//d/Tr20pQQKQwKItwdGhjWA1rKYDfUHuU6QyDzevzkFTETuJknKgj8hnB782doGDpDrTHvzOjklO41MnqxsneCbb3JtIFGgCvgA8NChQxg1ahTCw8MRERGBhx9+GGfPnvV0t4j8T1aWstIuBw/aXdpFKVdvA+cutnYaueIKZeeZOpVrA4kCVUCvAbx48SKGDh2KTp06YcOGDSgvL8fDDz8MSZKwaNEiT3ePyD8ozdgdOBD44QeXdsUd28C5i7Uaf7m59p+PawOJAktAB4CnTp3CsGHDsHDhQgQHBwMAXnrpJUyfPp0BIJEzNGsGlJfbbufkET9LGb7u2gbOXSztNGJrtxNzam61nJjItYFE/i6gp4Bbt26NxYsXG4M/AKiqqkKTJk082CsiP7BjhxxN2Ar+fvjB6cGftdInhsDI0ky0JMnFoJ2xDVx9OKtmn7V1gtb4yhQ4ETkuoEcAaxNCYPHixUhJSfF0V4h8l4u2cLOk5mhfQQEwa1bdNjWnN21tA/ff/3qmPqCzS9NY2gtYCV+YAicix7AMTA3z5s3D4sWLsW/fPoSFhdV5/sKFC7hw4YLxsU6nQ3R0dECnkRMZ3XYbsHat7XYObuFWk7mgyZKapU/WrDG/DdzYsfLom7vrA7qyNE3NAPnoUTnxw5acHPNTy0T+gmVgGAAarV27FikpKdi0aROuu+46s21mz56NOWaK1gbyLxAR/v5b2aK5ZcuASZOcdllLQZMthuCm9jrBsjIgJcX99QHdubevkhqCrA9IgYABIANAAMDOnTsRFxeHN998E5Os3KA4AkhUi5unew1sBU3WrFwJjBtn3/lcGRjl5srrFG1x1qicIXAGzE+BMwuYAgEDwABPAgGAkpISjBgxAo8++qjV4A8AgoODoVarTb6IAtKLLyoL/ioqXFLTz1Y9P2vMDVZ6sj6gu0vT2KohyOCPKDAEdBLImTNnMHLkSMTExODuu+/Gn3/+aXwuJiYGDRoE9MdDVNeFC0DjxrbbpaYC6eku60Z9giHDKJ65DF9P1gf0RGkaazUEiSgwBHSEs23bNuzYsQMA0LlzZ5PnCgsLERMT44FeEXkpD033mmNvMGToenq6+SDHk/UBbdXssxa4OsJSDUEiCgwBPQUcHx8PIYTZLwZ/RP/QapUFf4WFbgn+ANv1/GqzNb3pyfqASvb2tRS4EhHVV0AHgERkhWFriNGjrbfr109u68Z/NNUMmmyZM0eOTa2tbfN0EMZ1eUTkbswCdgCziMhvtWwJnDxpu52H//eRlSXX7rO0Y4a92bvm6gpGR8vBnzuCMEtb2BGRc/H+HeBrAImoll27gF69bLfLywOuv97l3bElIsL6dmk1s3eVrHfzdHIE1+URkbswACQimRcleSjliuxdBmFEFAi4BpAo0Cmt6afXe1XwB3g2e5eIyJcxACQKVKdPy4Hf889bb/f++3Lg56T9e53Jk9m7RES+jFPARIHIB6d7zTFk7yYny2/J3NZmLKFCRFSX9/2Tnohc53//Uxb86XReH/wZsIQKEZH9OAJIFAguXQIaNbLZrPqtJQh6aIobOuRcns7eJSLyNQwAifxd585AQYHNZhIENPOAjMt9c9SM2btERMpxCpjIX/30kzzdayP4uxylkCBP95aUyOvptFp3dJCIiDyFASCRvzFs4TZggNVmGXgMEgSO4nKTlwJAWpr1AsuW6PVAbi6wapX83Z5zOPJaIiKyDwNAIn8ybpyici0SBNJgfjPdmrtn2EOrlbcDHjwYGD9e/h4To2w00ZHXEhGR/bgGkMgfFBYCHTrYbrd7N1bt7gaMt93Unt0ztFp56rh24rBhSrlmNq6+Wo+8ojyUVpQiMiwSx3+Jwx1jVIpea4/a14lrGwdVELNCiIgABoBEvk9JWZdBg+R5VQCRZcpOq3T3DL0eSE01XzXGMBudliZn6a7Zr0VqdioO6w4b26gqNRBdMoD8JKuvtSejV5tf9zoatQYZCRlIivXBDBciIifjFDCRr3r5ZeVbuP0T/AHO3z0jLw84fNjy84Yp5Ze1WiRnJpsEZQCgb1ICpCQDsXXne+szHa3NN3+dEl0JkjOToc3nvDIREQNAIl9j2MLtueest/v6a7NbuBl2zwDqBoH12T1D0VSxpEfG/lQImBkmlP45lpAGSOYzP5ROR+ur9UjNNn8dw7G07DToq5lhQkSBjQEgkS+RJKB5c+ttQkLkwC8hwWITZ+6eoWiquF0eTlZZGSaUBBBeDLQzP9SndDo6ryivzshfTQICxbpi5BXZmeFCRORnuAaQqB70ejfvOpGZCdxxh+12Z84AoaGKTums3TMMU8olJebXAUoS0LxdKU4qOVlT06E+SZLPrXQ6urRC2VCh0nZERP6KI4BEdnJryZJLl+QoyFbw99ZbcvSlMPgzMOyeMW6c/L0+QaySKeXUexQO4VX+264+09GRYcquo7QdEZG/YgBIZAdDuZPaSQ8u2UGja1dF+/dCCOChh5x4YfvZmlJ+dkIcNGoNJFhKWpGgqowGDsXVea0909Fxba1fR4KEaHU04toqHFIkIvJTkhDmJm1ICZ1Oh/DwcJSXl0OtVnu6O+Rier080mcp49UwXVlY6OB0sNKafkeOKF8c5ybWpsYN2bkATJI0DMFaZnIWIo4nOTytbus6WSlZLAVDFOB4/2YA6BD+AgWW3Fx5uteWnBx5OtVuQgDNmgE6nfV2jz4KvPFGPS7geebq80Wro5GekO7UoMxd1yEi38T7N5NAiBRTWorEnh00jD74ALjvPtvtfPzfa0mxSUi8MtHlO3S46zpERL6KASCRQkpnW+2alT1xAoiIsN1u1y6gRw87Tuy9VEEqxMfE+811iIh8EZNAiBRy9g4aGDDAdvD36qvyqJ+fBH9EROQdOAJIpJCh3Elyshzs1ZyNtatkyddfA7fcYvuCen2dXTxIpq/Wc3qXiMgBDACJ7GAod5KaapoNrNHIwZ/VkiXnzimr07d7N9Ctm6Nd9VvmEjw0ag0yEjKY4EFEpBCzgB3ALKLAZfdOIJMmAStWWD/pQw/JBZ3JIkOJl9p7/bLECxHZg/dvBoAO4S8Q2fTLL8DVV9tud/48EBzs+v74MH21HjEZMRb3+pUgQaPWoDC1kNPBRGQV799MAiFyDb1eXhhoK/jbtEleTOih4E+vl+sbrlolf9frPdINRfKK8iwGf4Bc9LlYV4y8ojw39oqIyDcxACRytjlzgAY2ltcOGyYHfkoqS7uIW/c0doLSCmUFFpW2IyIKZEwCIXIWpVu4nT4NhIe7vDvWGPY0rr0AxLCnsb178LpDZJiyAotK2xERBTKOABI5SgigeXPbwd8nn8htFQR/+mo9cg/mYtXuVcg9mAt9tfPmZvV6OYvZ3Opfw7G0NOdMBztzijmubRw0ao0x4aM2CRKi1dGIa6u0ECMRUeDiCCCRI5YuBe6913qb6GigqEjxKV1d5iQvz7SEDQBA0gPt8oCmpRCVkSg+FIe8PFX99jT+h1ZrvlxORkb9RhdVQSpkJGQgOTMZEiSTTGBDUJiekM4EECIiBTgCSFQfJ07ISR62gr+iIruDv+TM5DrJDiW6EiRnJkOb7/gCvTp7FcdqgbQYYNJgIHm8/D0tBmv21/9ahinm2oGmYYq5vusMk2KTkJWShSh1lMlxjVrDEjBERHZgGRgHMI08QF1/PbB5s/U28+cDTz5p12ndVeYkN7dG7kmsFkhJBiBgMrMqJEACVtcjqNLr5WSSOqOM/5AkeSSwsFDBrimWrsGdQIjIAbx/cwqYSLnsbODmm223q+cWbvaUOYmPibf7/AaGPY0PH9EDCamoE/wBgCQgQUJadhoSr0y0K7gyO8VcgxBAcbHcrr5TzKoglUOfARFRoOMUMJEt58/Lw1a2gr9du+Topp7797qrzIlhT2O0zQPCD9cN/v5R37p6daaYHWxHRETOxwCQyJq77wZCQqy3mTJFDvx69HDoUu4sc5KUBKTNdE3AGamwe0rbERGR8zEAJDLn11/lUb/ly623O38eWLLEKZd0d5mTxCGuCTgNU8yShZFFSZITo+NYrYWIyGMYABLVZNjCrW9f6+2+/dbpW7gZypwAqBMEuqLMiasCTuMUM+oGgYbH6en1TwAhIiLHMQAkMnjhBZtbuJ3qcyNWrRTIVd3okn1z7Slz4miRZVcGnElJ8m4iUaZvAxqNd+4yQkQUaFgGxgFMI/cTx48DrVrZbHZVm1PYd6SZ8bEjRY1tsVXmxJlFls0Vno5WRyM9Id3hunp6vZztW1oqr/mLi+PIHxF5Hu/fDAAdwl8g72N3wHHvvfJuHlZsS/0Y/d64s87WaYbpTHePaFnax9eR/rCuHhEFEt6/GQA6hL9A3sWuUTGTasgWtGkDfVGJy4sa28MdRZaJiPwd799cA0h+QvHWY5WVQFiY7eCvqAgoKbGrqLE7eFt/iIjINzEAJJ+n18sjf+bGsg3H0tKA6ueel4O/ykrLJ/v8c/lF0dEAvK+osbf1h4iIfBO3giOfZ2tUrLvYhV3FvYCXrZzklluAr76qU7fE1UWN7V17xyLLRETkDAwAyeetWWP+eENcxE70QlfkWz/BwYNAu3ZmnzIUNS4pMT/CaFhzV5+ixuaybzVqDTISMixm37qyP0REFDg4BUw+TauViwrXNgVLcBHB1oO/d96RoygLwR/guqLG2nwtkjOTTYI/ACjRlSA5MxnafK3Z17HIMhEROQMDQPJZhrV/NcWgEAISluAhyy/s0QO4eBG4/35F13F2UWN9tR6p2akQqDuEZziWlp0GfbX5ys4sskxERI5iGRgHMI3cs2pWcpFQja9xM27CBusv+u03oHv3el3PWUWNcw/mYvAKG1nIAJ6Lew43drjR4rpAFlkmIqof3r+5BpB8mCHTNQmrsRrJ1hvPmQM8/7xD11OpgPh4h04BACitUJai+1LeS3gp7yWL6wKd1R8iIgo8nAImn9U25DgEJKvB30k0x/frKh0O/pwpMsy+FF1b6wKJiIjsxQCQfNN992Hg7db37x2E79Ar+iQGDm/ipk4pE9c2Dhq1BhIk242hbF0gERGRPRgAkm/57js53fWDDyw2eReTESQJ5Ek3eGVGrCpIhYwEOZXXniCwWFeMvCJu8UFERI5jAEi+obISCA+3uegtAsfxAN71+ozYpNgkZKVkIUodZbtxDUrXDxIREVnDAJC836xZ8hZuOp3FJvosLXJzBN5cGYGcHKCw0HuDP4Ok2CQcTD2InIk5eC7uOUWvsXf9IBERkTnMAibv9dtvQM+e1tvcfDPw1VdQBQUh3i2dco5/S7ioEBkZj5lxcVi+azlKdCVm6wNKkKBRaxDXllt8EBGR4zgCSN7n0iWgWzfbwV9hIbBuHRDkW7/GWi0QEyPXMBw/Xv7esYMK45qZXxdoeJyekG51n2AiIiKlfOvOSf7v7beBRo2AvXstt1myRN7CLSbGbd1yFq0WSE4GDpvuAIeSEmDBvUmY3rbuukCNWoOslCyL+wMTERHZizuBOICVxJ3o4EGgfXvrbbp3B375BWjY0C1dcja9Xo5Zawd/BpIkb+f25196/FiSh9KKUkSGRVrcCYSIiOqH92+OABrt3bsXI0eORPPmzT3dlcBSXS2v47MV/O3aJa8J9NHgD5DX/FkK/gB5ULO4GPhxswrxMfEY130c4mPiGfwREZHTMQAEkJubiz59+kCn04EDoq6hr9Yj92AuVu1ehdyDuXJBY61WLtKXnW35hbNmyZFRjx7u66yLlCqs4KK0HRERUX0xCxjA5Zdfjh07duDYsWMYNWqUp7vjd7T5WqRmp+KwTh7+ankGKHvNxovCw+XhsqZNXd9BN4lUWMFFaTsiIqL64ggggC5duqBr166e7oZf0uZrkZyZbAz+3v1SQfCXmwucPu1XwR8AxMXJa/wkC5t/SBIQHS23IyIiciWOANrhwoULuHDhgvGxzkphYpKnfVOzUyEg0OIscOJVGy+47z7gvffc0jdPUKmAjAw5C1iS5JltA0NQ6I1b1xERkf/hCKAd5s2bh/DwcONXdHS0p7vk1fKK8owjfytX22h87JhfB38GSUnyFnVRtXaA8/at64iIyL8wALTD008/jfLycuNXcXGxp7vk1WruWzvAwkc1OgVY9dtK4LLL3NQrz0tKkqve5OQAK1fC4a3r9Hp51nzVKvm7Xu/EzhIRkV/iFLAdgoODERwc7Olu+Iya+9aujgUm7fr3ueyOwC13AiIIeDQA97dVqYD4eMfPo9UCqamm5WU0GnmqmaOJRERkCQNAcpm4tnHQqDUo0ZXg/pECX18BBAng+3bAEbW8xVk097etN8OuIrUrF5WUyMc5pUxERJZwCphcRhWkQkaCvL9tVQMJmd2A/3X/N/gDuL9tfen18sifubKVhmNpaZwOJiIi8xgAkkslxSYhK4X72zqb0l1F8vLc1yciIvIdnAKuIT4+HqdPn/Z0N/xOUmwSEq9MRF4R97d1Fu4qQkREjmAASG6hCpL3tyXn4K4iRETkCE4BE/kg7ipCRESOYABI5IMMu4oAdYNA7ipCRES2MAAk8lHcVYSIiOqLawCJfFhSEpCYKGf7lpbKa/7i4jjyR0RE1jEAJPJxztpVhIiIAgengImIiIgCDANAIiIiogDDKWAKGHo918oREREBDAC9EgMV59Nq5b1za26fptHIpVSYLUtERIGGU8BeRqsFYmKAwYOB8ePl7zEx8nGqH60WSE6uu3duSYl8nJ8tEREFGgaAXoSBivPp9fLInxB1nzMcS0uT2xEREQUKBoBegoGKa+Tl1Q2oaxICKC6W2xEREQUKBoBegoGKa5SWOrcdERGRP2AA6CUYqLhGZKRz2xEREfkDBoBegoGKa8TFydm+kmT+eUkCoqPldkRERIGCAaCXYKDiGiqVXOoFqPvZGh6np7PMDhERBRYGgF6CgYrrJCUBWVlAVJTpcY1GPs46gEREFGgkIczlnZISOp0O4eHhKC8vh1qtdso5zRUsjo6Wgz8GKo5hgW0iIgJcc//2NQwAHeCqXyAGKq7Dz5aIiBgAcis4r6RSAfHxnu6F/+F2cERERDKuAaSAwF1WiIiI/sUAkPwed1khIiIyxQCQ/B53WSEiIjLFAJD8HndZISIiMsUAkPwed1khIiIyxQCQ/B53WSEiIjLFAJD8HndZISIiMsUAkAICt4MjIiL6FwtBU8BISgISE7kTCBEREQNACijcZYWIiIhTwEREREQBhwEgERERUYBhAEhEREQUYBgAEhEREQUYBoBEREREAYYBIBEREVGAYQBIREREFGAYABIREREFGAaARERERAGGASARERFRgGEASERERBRgGAASERERBRgGgEREREQBhgEgERERUYBhAEhEREQUYBgAEhEREQUYBoBEREREAaaBpztA5E301XrkFeWhtKIUkWGRiGsbB1WQytPdIiIicioGgET/0OZrkZqdisO6w8ZjGrUGGQkZSIpN8mDPiIiInItTwESQg7/kzGST4A8ASnQlSM5MhjZf66GeEREROR8DQAp4+mo9UrNTISDqPGc4lpadBn213t1dIyIicgkGgBTw8ory6oz81SQgUKwrRl5Rnht7RURE5DoMACnglVaUOrUdERGRt2MASAEvMizSqe2IiIi8HQNACnhxbeOgUWsgQTL7vAQJ0epoxLWNc3PPiIiIXIMBIAU8VZAKGQkZAFAnCDQ8Tk9IZz1AIiLyGwwAiQAkxSYhKyULUeook+MatQZZKVmsA0hERH5FEkLUrX1Biuh0OoSHh6O8vBxqtdrT3SEn4E4gRET+j/dv7gRCZEIVpEJ8TLynu0FERORSnAImIiIiCjAMAImIiIgCTMAHgFqtFl26dEHjxo3Rr18/bN++3dNdIiIiInKpgA4At2zZgvHjx+ORRx7Btm3b0L9/f9x00004duyYp7tGRERE5DIBnQU8atQoRERE4P333zce6927N0aNGoVZs2bZfD2ziIiIiHwP798BPgKYk5ODMWPGmBxLSUnBpk2bPNQjIiIiItcL2DIwZWVl0Ol06Nixo8nxjh07orCw0OxrLly4gAsXLhgf63Q6l/aRiIiIyBUCdgTw3LlzAIDw8HCT482bN8fZs2fNvmbevHkIDw83fkVHR7u8n0RERETOFrABYEhICACgvLzc5PipU6cQGhpq9jVPP/00ysvLjV/FxcUu7ycRERGRswXsFHBERATCwsJw4MABdOrUyXj8wIEDaN++vdnXBAcHIzg42PjYkD/DqWAiIiLfYbhvB3AebOAGgAAwZMgQZGVlYfjw4cZjmZmZSExMVPT6iooKAOBUMBERkQ+qqKiosxQsUAR0ADhjxgwMGTIEPXv2xKBBg/DBBx/g4MGDeOCBBxS9vk2bNiguLkZYWBgkSXJxb91Pp9MhOjoaxcXFAZsmrwQ/J+X4WSnHz0o5flbK8bOSCSFQUVGBNm3aeLorHhPQAeCAAQPw8ccf49lnn8W0adPQs2dPZGdn4/LLL1f0+qCgIGg0Ghf30vPUanVA/49CKX5OyvGzUo6flXL8rJTjZ1U3CTTQBHQACADJyclITk72dDeIiIiI3CZgs4CJiIiIAhUDQLIoODgYs2bNMsl8prr4OSnHz0o5flbK8bNSjp8VGQT0XsBEREREgYgjgEREREQBhgEgERERUYBhAEhEREQUYBgAkk379+9HcHAwYmJiPN0Vr/X222+jU6dOaNy4Mfr27YuNGzd6ukteSavVokuXLmjcuDH69euH7du3e7pLXunQoUMYNWoUwsPDERERgYcffhhnz571dLe82qRJkyBJEpYvX+7prni17OxsBAcHY//+/Z7uCnkYA0Cy6dFHHw2Igtf1tXTpUjz++ON4+umn8fPPP2P48OEYOXIkiouLPd01r7JlyxaMHz8ejzzyCLZt24b+/fvjpptuwrFjxzzdNa9y8eJFDB06FBcuXMCGDRuwcuVKbNiwATNmzPB017zWjz/+iMzMzIDe1UEJvV6P6dOnIzU1FZ07d/Z0d8jDGACSVVqtFtu3b8fzzz/v6a54rQYNGmDp0qW499570bNnT8ybNw/t2rXDhg0bPN01rzJ//nxMmDABjzzyCLp3746MjAy0bdsWS5Ys8XTXvMqpU6cwbNgwfPHFF+jXrx+GDx+Ol156CWvWrPF017ySXq/Hww8/jBkzZiAyMtLT3fFq7733Hk6ePImZM2d6uivkBRgAkkXnzp3DtGnT8PLLL6Nly5ae7o7X+s9//oM77rjD5Jher0eTJk081CPvlJOTgzFjxpgcS0lJwaZNmzzUI+/UunVrLF682KROW1VVFX+fLHj77bdx+vRpPPXUU57uilc7c+YMZs2ahfnz5yMsLMzT3SEvwACQLJo7dy4iIiJw//33e7orPmX9+vX4+++/cfPNN3u6K16jrKwMOp0OHTt2NDnesWNHFBYWeqhXvkEIgcWLFyMlJcXTXfE6ZWVlmDlzJtLT09G4cWNPd8ervfnmm6isrMTixYuhVqsxcOBA/Pzzz57uFnkQA0DChx9+iAYNGhi/OnXqhL/++gv//e9/sWjRIgQF8dfEwNxnVdOxY8cwefJkvPDCCwG/0XhN586dA1B38/XmzZszucGG+fPno6ioCE888YSnu+J1nnrqKfTr1w+JiYme7opXE0LgrbfeQlRUFKZPn47s7GzExsZi2LBhXKscwBp4ugPkeYmJidizZ4/xccOGDZGamoqxY8eif//+HuyZ9zH3WRlcuHABo0ePRu/evZGamuqJ7nmtkJAQAEB5eTkuu+wy4/FTp04hNDTUU93yemvXrsULL7yATZs2cdqulq1bt+KTTz7Bb7/95umueL2CggIcPnwYv/32G7p37w4AGDBgAHbv3o0PP/wQzz77rId7SJ7AAJAQHh5uMjJz+PBh/N///R+CgoLw4YcfApD/BVldXY0GDRpg48aNGDRokKe661G1P6ua7r33XpSXl2PdunWQJMnNPfNuERERCAsLw4EDB0xGTQ8cOID27dt7sGfea+fOnRg/fjyWLFmC6667ztPd8Trvvvsuzp8/j9jYWOMxvV6Pe++9Fx999BFLMdVQVlaGRo0aGYM/gz59+nAEMIAxAKQ6Wrdujd27d5sc+/777/Hyyy9j/fr1vGGbMWfOHHz77bfYunUrR2osGDJkCLKysjB8+HDjsczMTE7fmVFSUoIRI0bg0UcfxaRJkzzdHa80Z84cpKWlmRwbO3Ysxo0bh/Hjx3umU14qJiYGly5dwp49e9CtWzfj8V9//RW33367B3tGnsQAkOpo2LChyf8kAODgwYNmjxOwcuVKvPjii1ixYgUuXryIP//8EwCgVqvRqlUrD/fOe8yYMQNDhgxBz549MWjQIHzwwQc4ePAgHnjgAU93zaucOXMGI0eORExMDO6++27j7xMg38gbNOD/tgEgKioKUVFRJscaN26MqKgo/iO1ljZt2iAlJQV33HEHFixYgFatWuG9995DQUEBJk6c6OnukYfw/yREDnr33Xeh1+sxYcIEk+MTJ07krgQ1DBgwAB9//DGeffZZTJs2DT179kR2djYuv/xyT3fNq2zbtg07duwAgDrFegsLC7kjD9XL0qVL8eSTT2LixIk4c+YMrrvuOuTm5rJ2YgCThBDC050gIiIiIvdhfQ8iIiKiAMMAkIiIiCjAMAAkIiIiCjAMAImIiIgCDANAIiIiogDDAJCIiIgowDAAJCIiIgowDACJiIiIAgwDQCKif/z0009o0KABfv75Z5PjK1asQIsWLXDs2DEP9YyIyLm4EwgRUQ0PPvggfv75Z2zbtg0qlQoVFRXo3LkzXnjhBUyePNnT3SMicgqOABIR1TB//nyUlpZi0aJFAIAXXngBHTp0wH333efhnhEROU8DT3eAiMibhIeHIyMjA5MnT0bPnj2xePFibNmyBZIkebprREROwxFAIqJaUlJSMHDgQAwfPhyPPPIIunfv7ukuERE5FQNAIiIz+vTpg0uXLmHQoEGe7goRkdMxCYSIqJbCwkJcddVVSEhIwO+//45du3ahYcOGnu4WEZHTcASQiKiWtLQ03Hbbbfjkk09w5swZLFy40NNdIiJyKo4AEhHVkJ2djVGjRuH3339HTEwMPvvsM9xzzz34/fffERUV5enuERE5BQNAIqJ/XLx4Ed26dcOoUaPw6quvGo/Hx8ejdevW+PTTTz3YOyIi5+EUMBHRP15//XWcOnUKzz77rMnxN954A6tXr8bGjRs91DMiIufiCCARERFRgOEIIBEREVGAYQBIREREFGAYABIREREFGAaARERERAGGASARERFRgGEASERERBRgGAASERERBRgGgEREREQBhgEgERERUYBhAEhEREQUYBgAEhEREQWY/weM0UMsFtVjxwAAAABJRU5ErkJggg==",
      "text/html": [
       "\n",
       "            <div style=\"display: inline-block;\">\n",
       "                <div class=\"jupyter-widgets widget-label\" style=\"text-align: center;\">\n",
       "                    Figure\n",
       "                </div>\n",
       "                <img src='' width=640.0/>\n",
       "            </div>\n",
       "        "
      ],
      "text/plain": [
       "Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np  \n",
    "import matplotlib.pyplot as plt  \n",
    "from sklearn.linear_model import LinearRegression  \n",
    "from sklearn.model_selection import train_test_split  \n",
    "  \n",
    "# 创建一些模拟数据  \n",
    "np.random.seed(0)  \n",
    "X = 2.5 * np.random.randn(100) + 1.5  # 100个数据点  \n",
    "res = 0.5 * np.random.randn(100)     # 噪声  \n",
    "y = 2 + 0.3 * X + res  \n",
    "  \n",
    "# 划分训练集和测试集（这里仅为示例，实际上线性回归不需要这样做）  \n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)  \n",
    "  \n",
    "# 创建线性回归模型  \n",
    "model = LinearRegression()  \n",
    "  \n",
    "# 训练模型  \n",
    "model.fit(X_train.reshape(-1, 1), y_train)  \n",
    "  \n",
    "# 使用模型进行预测  \n",
    "y_pred = model.predict(X_test.reshape(-1, 1))  \n",
    "  \n",
    "# 设置matplotlib的字体参数，确保中文能够正确显示  \n",
    "# 注意：这里使用的是'MiSans'字体，您需要确保这个字体在您的系统中是可用的  \n",
    "# 如果您使用的是其他操作系统或字体，请相应地更改字体名称  \n",
    "plt.rcParams['font.sans-serif'] = ['MiSans']  # 指定默认字体为黑体  \n",
    "plt.rcParams['axes.unicode_minus'] = False  # 解决保存图像时负号'-'显示为方块的问题  \n",
    " \n",
    "# 绘制结果  \n",
    "plt.scatter(X_train, y_train, color='blue', label='Training data')  \n",
    "plt.scatter(X_test, y_test, color='green', label='Test data')  \n",
    "plt.plot(X_test, y_pred, color='red', linewidth=3, label='Predicted regression line')  \n",
    "plt.xlabel('X')  \n",
    "plt.ylabel('y')  \n",
    "plt.title('线性回归数据拟合效果')  \n",
    "plt.legend()  \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b0f34f2",
   "metadata": {},
   "source": [
    "### 损失函数\n",
    "\n",
    "- 在我们开始考虑如何用模型拟合（fit）数据之前，我们需要确定一个拟合程度的度量。\n",
    "- 损失函数（loss function）能够量化目标的实际值与预测值之间的差距。\n",
    "- 通常我们会选择非负数作为损失，且数值越小表示损失越小，完美预测时的损失为0。\n",
    "- 回归问题中最常用的损失函数是平方误差函数。\n",
    "- 由于平方误差函数中的二次方项，估计值$\\hat{y}(i)$和观测值$y(i)$之间较大的差异将导致更大的损失。\n",
    "- 为了度量模型在整个数据集上的质量，我们需计算在训练集n个样本上的损失均值（也等价于求和）。\n",
    "- MSE均方差损失函数（L2损失函数）表示如下：\n",
    "\n",
    "$L(w,b) = \\frac{1}{n} \\underset{i=1}{\\overset{n}{\\sum}} = \\frac{1}{n} \\sum_{i=1}^{n} \\frac{1}{2} ( w^T x^{(i)} + b - y^{(i)} )^2$\n",
    "\n",
    "- 在训练模型时，我们希望寻找一组参数（w∗，b∗），这组参数能最小化在所有训练样本上的总损失。\n",
    "\n",
    "$w*,b* = arg\\_min_{w,b} L(w,b)$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88488b0f-7eca-4542-9acd-736accbfc4dd",
   "metadata": {},
   "source": [
    "### 训练和预测\n",
    "\n",
    "线性回归整个算法训练流程：\n",
    "\n",
    "- 初始化参数\n",
    "\n",
    "- 重复以下训练，直到完成\n",
    "\n",
    "计算梯度 $g \\leftarrow \\partial (w,b)^{\\frac{1}{\\left| B \\right|}\\sum_{icB}l(x^i,y^i,w,b)}$\n",
    "\n",
    "更新参数 $(w,b) \\leftarrow (w,b) - \\eta g$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e2ef5c2-7d98-465a-a2f3-aecc963cc611",
   "metadata": {},
   "source": [
    "**优缺点**\n",
    "\n",
    "优点：\n",
    "\n",
    "1. 善于获取数据集中的线性关系；\n",
    "\n",
    "2. 适用于在已有了一些预先定义好的变量并且需要一个简单的预测模型的情况下使用；\n",
    "\n",
    "3. 训练速度和预测速度较快；\n",
    "\n",
    "4. 在小数据集上表现很好；\n",
    "\n",
    "5. 结果可解释，并且易于说明；\n",
    "\n",
    "6. 当新增数据时，易于更新模型；\n",
    "\n",
    "7. 不需要进行参数调整（正则化线性模型需要调整正则化参数）；\n",
    "\n",
    "8. 不需要特征缩放（正则化线性模型需要特征缩放）；\n",
    "\n",
    "缺点：\n",
    "\n",
    "1. 不适用于非线性数据；\n",
    "\n",
    "2. 预测精确度较低；\n",
    "\n",
    "3. 可能会出现过度拟合（正则化模型可以抵消这个影响）；\n",
    "\n",
    "4. 分离信号和噪声的效果不理想，在使用前需要去掉不相关的特征；\n",
    "\n",
    "5. 不了解数据集中的特征交互；"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ec70095",
   "metadata": {},
   "source": [
    "## 多层感知机\n",
    "\n",
    "- 最简单的深度网络称为多层感知机（MLP）。\n",
    "- 多层感知机由多层神经元组成，每一层与它的上一层相连，从中接收输入。\n",
    "- 同时每一层也与它的下一层相连，影响当前层的神经元。\n",
    "- 多层感知机在输出层和输入层之间增加一个或多个全连接隐藏层，并通过激活函数转换隐藏层的输出。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2917a6f",
   "metadata": {},
   "source": [
    "### 数学模型\n",
    "\n",
    "MLP（FNN）工作原理：多层感知机首先是输入层，中间是隐藏层，最后是输出层。\n",
    "\n",
    "$H = \\sigma (XW^{(1)} + b^{(1)})$\n",
    "\n",
    "$O = HW^{(2)} + b^{(2)}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cfcda5b5-7376-4aef-9166-9856ee82dbe4",
   "metadata": {},
   "source": [
    "<img src=\"drawio/A02.drawio.png\">\n",
    "\n",
    "一个单隐藏层的多层感知机，具有5个隐藏层"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d3caed8",
   "metadata": {},
   "source": [
    "### 隐藏层\n",
    "\n",
    "- 激活函数（activation function）通过计算加权和并加上偏置来确定神经元是否应该被激活，它们将输入信号转换为输出的可微运算。\n",
    "- 大多数激活函数都是非线性的。\n",
    "- 常用的激活函数包括ReLU函数、sigmoid函数和tanh函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49e507bc",
   "metadata": {},
   "source": [
    "#### ReLU函数\n",
    "\n",
    "- ReLU提供了一种非常简单的非线性变换。\n",
    "- 给定元素x，ReLU函数被定义为该元素与0的最大值。\n",
    "- ReLU函数通过将相应的活性值设为0，仅保留正元素并丢弃所有负元素。\n",
    "- 当输入为负时，ReLU函数的导数为0，而当输入为正时，ReLU函数的导数为1。\n",
    "- ReLU函数之所以是最受欢迎的激活函数，原因是它求导表现得特别好：要么让参数消失，要么让参数通过。\n",
    "- 这使得优化表现得更好，并且ReLU减轻了困扰以往神经网络的梯度消失问题。\n",
    "\n",
    "$ReLU(x) = max(x, 0)$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52a617f5",
   "metadata": {},
   "source": [
    "#### sigmoid函数\n",
    "\n",
    "- 对于一个定义域在R中的输入，sigmoid函数将输入变换为区间（0，1）上的输出。\n",
    "- 因此，sigmoid通常称为挤压函数（squashing function）：\n",
    "- 它将范围（-inf, inf）中的任意输入压缩到区间（0，1）中的某个值：\n",
    "\n",
    "$sigmoid(x) = \\frac{1}{1+exp(-x)}$\n",
    "\n",
    "- 注意，当输入为0时，sigmoid函数的导数达到最大值0.25；\n",
    "- 而输入在任一方向上越远离0点时，导数越接近0。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f1e0a91-cb24-45e1-a167-caa04cdf77b4",
   "metadata": {},
   "source": [
    "#### tanh函数\n",
    "\n",
    "与sigmoid函数类似，tanh（双曲正切）函数也能将其输入压缩转换到区间（-1，1）上。tanh函数的公式如下：\n",
    "\n",
    "$tanh(x) = \\frac{1-exp(-2x)}{1+exp(-2x)}$\n",
    "\n",
    "当输入接近0时，tanh函数的导数接近最大值1。与我们在sigmoid函数图像中看到的类似，输入在任一方向上越远离0点，导数越接近0。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36010a77-d901-4f6f-b08b-4db38a7d9588",
   "metadata": {},
   "source": [
    "### 训练和预测\n",
    "\n",
    "1. 初始化模型参数\n",
    "\n",
    "2. 定义激活函数\n",
    "\n",
    "3. 定义模型\n",
    "\n",
    "4. 定义损失函数\n",
    "\n",
    "5. 进行模型训练"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d264577f-ac85-4d70-9205-e21d9a41a451",
   "metadata": {},
   "source": [
    "### 欠拟合和过拟合\n",
    "\n",
    "将模型在训练数据上拟合的比在潜在分布中更接近的现象称为过拟合（overfitting），用于对抗过拟合的技术称为正则化（regularization）。\n",
    "\n",
    "由于我们的训练和验证误差之间的泛化误差很小，我们有理由相信可以用一个更复杂的模型降低训练误差。\n",
    "这种现象被称为欠拟合（underfitting）。\n",
    "\n",
    "训练误差（training error）是指，模型在训练数据集上计算得到的误差。\n",
    "泛化误差（generalization error）是指，模型应用在同样从原始样本的分布中抽取的无限多数据样本时，模型误差的期望。\n",
    "\n",
    "两种防止过拟合的方法：权重衰减和暂退法（dropout）。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "104ced10-a12e-4d8c-bdf2-29d8bda6792a",
   "metadata": {},
   "source": [
    "### 数值稳定性：\n",
    "\n",
    "梯度消失和梯度爆炸是深度网络中常见的问题。\n",
    "在参数初始化时需要非常小心，以确保梯度和参数可以得到很好的控制。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6abefef-40fb-4fb4-bdb6-06cdea298f75",
   "metadata": {},
   "source": [
    "### 优缺点\n",
    "\n",
    "优点：可以学习非线性模型，并且可以实时学习。\n",
    "\n",
    "缺点：多层感知机随着层数越多，会出现过拟合（权重衰减和暂退法可以预防），参数难以调试，梯度弥散问题。\n",
    "再图像识别时，将图像数据展平成一维向量为忽略了每个图像的空间结构信息，且各层均为连接层，计算量巨大。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94e6fb10-eaf6-4192-954e-e0407c232788",
   "metadata": {},
   "source": [
    "## 卷积神经网络\n",
    "\n",
    "卷积神经网络（convolutional neural network，CNN）是一类强大的、为处理图像数据而设计的神经网络。\n",
    "基于卷积神经网络架构的模型在计算机视觉领域中已经占主导地位，当今几乎所有的图像识别、目标检测或语义分割相关的学术竞赛和商业应用都以这种方法为基础。\n",
    "卷积神经网络需要的参数少于全连接架构的网络，而且卷积也很容易用GPU并行计算。\n",
    "因此卷积神经网络除了能够高效地采样从而获得精确的模型，还能够高效地计算。\n",
    "\n",
    "构成卷积网络主干的基本元素包括：卷积层本身、填充（padding）和步幅（stride）、用于在相邻区域汇聚信息的汇聚层（pooling）、在每一层中多通道（channel）。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4b419f2-6744-4ff2-973a-7c9e813cabbd",
   "metadata": {},
   "source": [
    "### 数学模型\n",
    "\n",
    "${\\left [ H \\right ]}_{i,j} = u + \\sum_{a=-∆}^{∆} \\sum_{a=-∆}^{∆} {\\left [ V \\right ]}_{a,b} {\\left [ X \\right ]}_{i+a,j+b}$\n",
    "\n",
    "是一个卷积层（convolutional layer），而卷积神经网络是包含卷积层的一类特殊的神经网络。\n",
    "在深度学习中，V被称为卷积核（convolution kernel）或者滤波器（filter），亦或简单地称之为该卷积层的权重，通常该权重是可学习的参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "282e0b1d-f3e4-4726-b662-4ee4203f0ae8",
   "metadata": {},
   "source": [
    "### 现代CNN类型\n",
    "\n",
    "（1）深度卷积神经网络（AlexNet）：AlexNet由八层组成：五个卷积层、两个全连接隐藏层和一个全连接输出层。\n",
    "AlexNet使用ReLU而不是sigmoid作为其激活函数。\n",
    "\n",
    "（2）使用块的网络（VGG）：VGG网络可以分为两部分：第一部分主要由卷积层和汇聚层组成，第二部分由全连接层组成。\n",
    "VGG网络有5个卷积块，其中前两个块各有一个卷积层，后三个块各包含两个卷积层。\n",
    "\n",
    "（3）网络中的网络（NiN）：NiN块以一个普通卷积层开始，后面是两个1×1的卷积层。\n",
    "这两个1×1卷积层充当带有ReLU激活函数的逐像素全连接层。\n",
    "第一层的卷积窗口形状通常由用户设置。\n",
    "随后的卷积窗口形状固定为1×1。\n",
    "\n",
    "NiN使用窗口形状为11×11、5×5和3×3的卷积层，输出通道数量与AlexNet中的相同。\n",
    "每个NiN块后有一个最大汇聚层，汇聚窗口形状为3×3，步幅为2。\n",
    "\n",
    "NiN和AlexNet之间的一个显著区别是NiN完全取消了全连接层。\n",
    "相反，NiN使用一个NiN块，其输出通道数等于标签类别的数量。\n",
    "最后放一个全局平均汇聚层（global average pooling layer），生成一个对数几率（logits）。\n",
    "NiN设计的一个优点是，它显著减少了模型所需参数的数量。\n",
    "然而，在实践中，这种设计有时会增加训练模型的时间。\n",
    "\n",
    "（4）含并行连结的网络（GoogLeNet）：在GoogLeNet中，基本的卷积块被称为Inception块（Inception block）。\n",
    "Inception块由四条并行路径组成。\n",
    "前三条路径使用窗口大小为1×1、3×3和5×5的卷积层，从不同空间大小中提取信息。\n",
    "中间的两条路径在输入上执行1×1卷积，以减少通道数，从而降低模型的复杂性。\n",
    "第四条路径使用3×3最大汇聚层，然后使用1×1卷积层来改变通道数。\n",
    "这四条路径都使用合适的填充来使输入与输出的高和宽一致，最后我们将每条线路的输出在通道维度上连结，并构成Inception块的输出。\n",
    "在Inception块中，通常调整的超参数是每层输出通道数。\n",
    "\n",
    "GoogLeNet一共使用9个Inception块和全局平均汇聚层的堆叠来生成其估计值。\n",
    "Inception块之间的最大汇聚层可降低维度。\n",
    "第一个模块类似于AlexNet和LeNet，Inception块的组合从VGG继承，全局平均汇聚层避免了在最后使用全连接层。\n",
    "\n",
    "（5）残差网络（ResNet）：ResNet沿用了VGG完整的3×3卷积层设计。\n",
    "残差块里首先有2个有相同输出通道数的3×3卷积层。\n",
    "每个卷积层后接一个批量规范化层和ReLU激活函数。\n",
    "然后我们通过跨层数据通路，跳过这2个卷积运算，将输入直接加在最后的ReLU激活函数前。\n",
    "这样的设计要求2个卷积层的输出与输入形状一样，从而使它们可以相加。\n",
    "如果想改变通道数，就需要引入一个额外的1×1卷积层来将输入变换成需要的形状后再做相加运算。\n",
    "\n",
    "ResNet的前两层跟之前介绍的GoogLeNet中的一样：在输出通道数为64、步幅为2的7×7卷积层后，接步幅为2的3×3的最大汇聚层。\n",
    "不同之处在于ResNet每个卷积层后增加了批量规范化层。\n",
    "GoogLeNet在后面接了4个由Inception块组成的模块。\n",
    "ResNet则使用4个由残差块组成的模块，每个模块使用若干个同样输出通道数的残差块。\n",
    "第一个模块的通道数同输入通道数一致。\n",
    "由于之前已经使用了步幅为2的最大汇聚层，所以无须减小高和宽。\n",
    "之后的每个模块在第一个残差块里将上一个模块的通道数翻倍，并将高和宽减半。\n",
    "\n",
    "（6）稠密连接网络（DenseNet）：稠密网络主要由2部分构成：稠密块（dense block）和过渡层（transition layer）。\n",
    "前者定义如何连接输入和输出，而后者则控制通道数量，使其不会太复杂。\n",
    "ResNet与 DenseNet在跨层连接上的主要区别：使用相加和使用连结。\n",
    "\n",
    "一个稠密块由多个卷积块组成，每个卷积块使用相同数量的输出通道。\n",
    "然而，在前向传播中，我们将每个卷积块的输入和输出在通道维上连结。\n",
    "\n",
    "由于每个稠密块都会带来通道数的增加，使用过多则会过于复杂化模型。\n",
    "而过渡层可以用来控制模型复杂度。\n",
    "它通过1 × 1卷积层来减小通道数，并使用步幅为2的平均汇聚层减半高和宽，从而进一步降低模型复杂度。\n",
    "\n",
    "DenseNet首先使用同ResNet一样的单卷积层和最大汇聚层。\n",
    "接下来，类似于ResNet使用的4个残差块，DenseNet使用的是4个稠密块。\n",
    "与ResNet类似，我们可以设置每个稠密块使用多少个卷积层。\n",
    "接下来，类似于ResNet使用的4个残差块，DenseNet使用的是4个稠密块。\n",
    "与ResNet类似，我们可以设置每个稠密块使用多少个卷积层。\n",
    "与ResNet类似，最后接上全局汇聚层和全连接层来输出结果。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d539e7e-eab6-4961-b597-1ca657049daf",
   "metadata": {},
   "source": [
    "## 循环神经网络\n",
    "\n",
    "循环神经网络（recurrent neural network，RNN）通过引入状态变量存储过去的信息和当前的输入，从而可以确定当前的输出。\n",
    "所以循环神经网络可以更好地处理序列信息。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99edccb1-117f-41b7-beb4-26b860e6f0ca",
   "metadata": {},
   "source": [
    "### 数学模型\n",
    "\n",
    "（1）无隐状态的神经网络\n",
    "\n",
    "设隐藏层的激活函数为ϕ，给定一个小批量样本$X \\in R^{nxd}$，其中批量大小为n，输入维度为d，则隐藏层的输出$H \\in R^{nxh}$通过下式计算：\n",
    "\n",
    "$H = \\phi (WH_{xh} + b_h)$\n",
    "\n",
    "我们拥有的隐藏层权重参数为$W_{xh} \\in R^{dxh}$，偏置参数为$b_h \\in R^{1xh}$，以及隐藏单元的数目为h。\n",
    "因此求和时可以应用广播机制。\n",
    "接下来，将隐藏变量H用作输出层的输入。\n",
    "输出层由下式给出：\n",
    "\n",
    "$O = HW_{hq} + b_q$\n",
    "\n",
    "其中，$O \\in R^{nxq}$是输出变量，$W_{hq} \\in R^{hxq}$是权重参数，$b_q \\in R^{1xq}$是输出层的偏置参数。\n",
    "\n",
    "（2）有隐状态的循环神经网络\n",
    "\n",
    "假设我们在时间步t有小批量输入$X^t \\in R^{nxd}$。换⾔之，对于n个序列样本的小批量，Xt的每一行对应于来自该序列的时间步t处的一个样本。\n",
    "接下来，用$H^t \\in R^{nxh}$表示时间步t的隐藏变量。\n",
    "与多层感知机不同的是，我们在这里保存了前一个时间步的隐藏变量$H^{t-1}$，并引入了一个新的权重参数$W_{hh} \\in R^{hxh}$，来描述如何在当前时间步中使用前一个时间步的隐藏变量。\n",
    "具体地说，当前时间步隐藏变量由当前时间步的输入与前一个时间步的隐藏变量一起计算得出：\n",
    "\n",
    "$H_t = \\phi (X_tW_{xh} + H_{t-1}W_{hh} + b_h)$\n",
    "\n",
    "对于时间步t，输出层的输出类似于多层感知机中的计算：\n",
    "\n",
    "$O_t = H_tW_{hq} + b_q$\n",
    "\n",
    "其中循环神经网络的参数包括隐藏层的权重 $W^{xh} \\in R^{dxh}$, $W_{hh} \\in R^{hxh}$ 和偏置$b^h \\in R^{1xh}$，以及输出层的权重$W_{hq} \\in R^{hxq}$ 和偏置$b_q \\in R^{1xq}$。\n",
    "\n",
    "### 训练和预测\n",
    "\n",
    "训练：\n",
    "\n",
    "1. 序列数据的不同采样方法（随机采样和顺序分区）将导致隐状态初始化的差异。\n",
    "\n",
    "2. 我们在更新模型参数之前裁剪梯度。这样的操作的目的是：即使训练过程中某个点上发生了梯度爆炸，也能保证模型不会发散。\n",
    "\n",
    "3. 我们用困惑度来评价模型。\n",
    "\n",
    "预测：\n",
    "\n",
    "1. 预热（warm-up）期：⾸先定义预测函数来生成prefix之后的新字符，其中的prefix是一个用户提供的包含多个字符的字符串。\n",
    "   在循环遍历prefix中的开始字符时，我们不断地将隐状态传递到下一个时间步，但是不生成任何输出。\n",
    "\n",
    "2. 在此期间模型会自我更新（例如，更新隐状态），但不会进行预测。\n",
    "   预热期结束后，隐状态的值通常比刚开始的初始值更适合预测，从而预测字符并输出它们。\n",
    "\n",
    "### 现代RNN\n",
    "\n",
    "（1）门控循环单元（GRU）：\n",
    "\n",
    "门控循环单元与普通的循环神经网络之间的关键区别在于：前者支持隐状态的门控。\n",
    "这意味着模型有专门的机制来确定应该何时更新隐状态，以及应该何时重置隐状态。\n",
    "例如，如果第一个词元非常重要，模型将学会在第一次观测之后不更新隐状态。\n",
    "同样，模型也可以学会跳过不相关的临时观测。最后，模型还将学会在需要的时候重置隐状态。\n",
    "\n",
    "（2） 长短期记忆网络（LSTM）：\n",
    "\n",
    "长短期记忆网络的设计灵感来自于计算机的逻辑门。\n",
    "长短期记忆网络引入了记忆元（memory cell），或简称为单元（cell）为了控制记忆元，我们需要许多门。\n",
    "其中一个门用来从单元中输出条目，我们将其称为输出门（output gate）。\n",
    "另外一个门用来决定何时将数据读入单元，我们将其称为输入门（input gate）。\n",
    "我们还需要一种机制来重置单元的内容，由遗忘门（forget gate）来管理，这种设计的动机与门控循环单元相同，能够通过专用机制决定什么时候记忆或忽略隐状态中的输入。\n",
    "\n",
    "（3） 深度循环神经网络：\n",
    "\n",
    "深度循环神经网络可以将多层循环神经网络堆叠在一起，通过对几个简单层的组合，产生了一个灵活的机制。\n",
    "特别是，数据可能与不同层的堆叠有关。\n",
    "在深度循环神经网络中，隐状态的信息被传递到当前层的下一时间步和下一层的当前时间步。\n",
    "深度循环神经网络需要大量的调参（如学习率和修剪）来确保合适的收敛，模型的初始化也需要谨慎。\n",
    "\n",
    "（5） 双向循环神经网络：\n",
    "\n",
    "双向循环神经网络（bidirectional RNNs）添加了反向传递信息的隐藏层，以便更灵活地处理此类信息。\n",
    "双向循环神经网络的一个关键特性是：使用来自序列两端的信息来估计输出。\n",
    "也就是说，我们使用来自过去和未来的观测信息来预测当前的观测。\n",
    "\n",
    "存在的缺点：\n",
    "\n",
    "    1. 在训练期间，我们能够利用过去和未来的数据来估计现在空缺的词；而在测试期间，我们只有过去的数据，因此精度将会很差。\n",
    "\n",
    "    2. 双向循环神经网络的计算速度非常慢。\n",
    "       其主要原因是网络的前向传播需要在双向层中进行前向和后向递归，并且网络的反向传播还依赖于前向传播的结果。因此，梯度求解将有一个非常长的链。\n",
    "\n",
    "（6） 编码器-解码器架构：\n",
    "\n",
    "机器翻译是序列转换模型的一个核⼼问题，其输入和输出都是长度可变的序列。\n",
    "为了处理这种类型的输入和输出，我们可以设计一个包含两个主要组件的架构：第一个组件是一个编码器（encoder）：它接受一个长度可变的序列作为输入，并将其转换为具有固定形状的编码状态。\n",
    "第二个组件是解码器（decoder）：它将固定形状的编码状态映射到长度可变的序列。\n",
    "这被称为编码器-解码器（encoder-decoder）架构。\n",
    "\n",
    "（7） 序列到序列学习（seq2seq）：\n",
    "\n",
    "序列到序列学习将使用两个循环神经网络的编码器和解码器，输入序列的信息被编码到循环神经网络编码器的隐状态中。\n",
    "为了连续生成输出序列的词元，独立的循环神经网络解码器是基于输入序列的编码信息和输出序列已经看见的或者生成的词元来预测下一个词元。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24ae2cdb-684e-4b4c-8160-305239e0f07f",
   "metadata": {},
   "source": [
    "## 优化算法\n",
    "\n",
    "对于深度学习问题， 我们通常会先定义损失函数。\n",
    "一旦我们有了损失函数，我们就可以使用优化算法来尝试最小化损失。\n",
    "在优化中，损失函数通常被称为优化问题的目标函数。\n",
    "\n",
    "（1）梯度下降（批量梯度下降）：对样本集中所有样本计算梯度后求平均，更新参数。\n",
    "\n",
    "（2）随机梯度下降：从样本集中随机选取一个样本计算梯度,更新参数。\n",
    "\n",
    "（3）小批量随机梯度下降：从样本集中随机选取多个样本计算梯度求平均,更新参数。\n",
    "\n",
    "（4）动量法：是传统梯度下降法（SGD）的一种扩展，它比SGD更高效。\n",
    "动量法又被称作基于动量的梯度下降法（SGD with momentum），是一种使梯度向量向相关方向加速变化、最终实现加速收敛的方法。\n",
    "\n",
    "（5）AdaGrad算法：利用梯度的大小作为调整进度速率的手段：用较小的学习率来补偿带有较大梯度的坐标。\n",
    "\n",
    "AdaGrad算法的优点：首先，我们不再需要决定梯度何时算足够大。\n",
    "其次，它会随梯度的大小自动变化。\n",
    "通常对应于较大梯度的坐标会显著缩小，而其他梯度较小的坐标则会得到更平滑的处理。\n",
    "\n",
    "AdaGrad算法的缺点：AdaGrad算法有时在降低学习率方面可能过于剧烈。\n",
    "\n",
    "（6）RMSProp算法：RMSProp算法与AdaGrad算法非常相似，因为两者都使用梯度的平方来缩放系数。\n",
    "RMSProp算法与动量法都使用泄漏平均值。但是，RMSProp算法使用该技术来调整按系数顺序的预处理器。\n",
    "\n",
    "（7）Adadelta算法：Adadelta是AdaGrad、RMSProp的另一种变体，使用参数的变化量来代替学习率，因此广义上认为Adadelta没有学习率。\n",
    "\n",
    "（8）Adam算法：使用指数加权移动平均值来估算梯度的动量（momentum）和二阶矩（moment），在此基础上更新网络参数。\n",
    "\n",
    "（9）学习率调度器：单因子调度器、多因子调度器、余弦调度器、预热。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fecb15a-55bc-4952-9469-97c62bf375f0",
   "metadata": {},
   "source": [
    "## 自定义网络实现\n",
    "\n",
    "实现自定义块需要提供以下基本功能：\n",
    "\n",
    "1. 将输入数据作为其前向传播函数的参数。\n",
    "2. 通过前向传播函数来生成输出。请注意，输出的形状可能与输入的形状不同。\n",
    "3. 计算其输出关于输入的梯度，可通过其反向传播函数进行访问。通常这是自动发生的。\n",
    "4. 存储和访问前向传播计算所需的参数。\n",
    "5. 根据需要初始化模型参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84cd69fa-e707-4ae6-9a83-73a25f71b8ea",
   "metadata": {
    "tags": []
   },
   "source": [
    "# 深度学习应用"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d173e23-b9c4-45b8-8628-2f1dd5f9b8fb",
   "metadata": {},
   "source": [
    "## 单发多框目标检测\n",
    "\n",
    "模型由一个基础网络块和多个多尺度特征块组成。\n",
    "每个特征块将上一层特征图的高和宽缩小（如减半），因而相同大小的锚框层数越深在原始输入图像上的感受野越大（可检测的对象越大）。\n",
    "每个块生成一定数量的锚框，并通过类别预测层和边界框预测层预测其类别和偏移量来检测大小不同的目标。\n",
    "\n",
    "### 类别预测层\n",
    "\n",
    "设目标类别的数量为q。这样一来，锚框有q+1个类别，其中0类是背景。在某个尺度下，设特征图的高和宽分别为h和w。\n",
    "如果以其中每个像素为中心生成a个锚框，那么我们需要对hwa个锚框进行分类。\n",
    "\n",
    "类别预测层是一个填充为1，卷积核为3×3，输出通道数为a（q+1）的卷积层，\n",
    "可使输入和输出的宽度和高度保持不变，第i（q+1）+j（0≤j≤q）个输出通道中的第x行y列元素值代表了输入中第x行y列像素值生成的第i个锚框的类别索引为j时的预测值。\n",
    "\n",
    "### 边界框预测层\n",
    "\n",
    "边界框预测层的设计与类别预测层类似。不同的是，需要为每个锚框预测4个偏移量，而不是q+1个类别。\n",
    "\n",
    "### 连结多尺度的预测\n",
    "\n",
    "在不同的尺度下，特征图的形状或以同一单元为中心的锚框的数量可能会有所不同。因此，不同块的预测输出的形状可能会有所不同。\n",
    "\n",
    "为了提高计算效率，我们将模型每个块的预测结果转成二维的（批量大小，高×宽×通道数）的格式，再在维度1上连结。\n",
    "\n",
    "### 高和宽减半块\n",
    "\n",
    "为了在多个尺度下检测目标，模型中多数特征块采用高和宽减半块，即多数特征块将来自上一模块的输入特征图的高度和宽度减半。\n",
    "更具体地说，每个高和宽减半块由两个填充为1的3×3的卷积层、以及步幅为2的2×2最大汇聚层组成。\n",
    "\n",
    "### 基本网络块\n",
    "\n",
    "基本网络块用于从输入图像中抽取特征。为了计算简洁，模型采用了一个小的基础网络，该网络串联3个高和宽减半块，并逐步将通道数翻倍。\n",
    "\n",
    "### 完整的模型\n",
    "\n",
    "完整的单发多框检测模型由五个模块组成。每个块生成的特征图既用于生成锚框，又用于预测这些锚框的类别和偏移量。\n",
    "在这五个模块中，第一个是基本网络块，第二个到第四个是高和宽减半块，最后一个模块使用全局最大池化将高度和宽度都降到1。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8914a79e-7937-447b-8e08-7fba6b79e063",
   "metadata": {},
   "source": [
    "## 语义分割\n",
    "\n",
    "### 语义分割基本原理\n",
    "\n",
    "（1）语义分割：识别并理解图像中每一个像素的内容（对图像每一像素分类），其语义区域的标注和预测是像素级的。\n",
    "与目标检测相比，语义分割标注的像素级的边框显然更加精细。\n",
    "\n",
    "（2）模型：采用全卷积网络构建模型，引入1×1卷积层将输出通道数调整为待分割图像一个像素的可能类别总数，引入转置卷积层将输出图像尺寸调整得和输入图像尺寸一样。\n",
    "\n",
    "### 图像分割和实例分割（语义分割介于二者之间）\n",
    "\n",
    "图像分割将图像划分为若干组成区域,这类问题的方法通常利用图像中像素之间的相关性。\n",
    "它在训练时不需要有关困像像素的标签信息，在预测时也无法保证分割出的区域具有我们希望得到的语义。\n",
    "例如猫狗图像，图像分割可能会将猫狗分为两个区域：一个覆盖以黑色为主的嘴和眼睛，另一个覆盖以黄色为主的其余部分身体。\n",
    "\n",
    "实例分割也叫同时检测并分割，它研究如何识别图像中各个目标实例的像素级区域。\n",
    "与语义分割不同,实例分割不仅需要区分语义，还要区分不同的目标实例。\n",
    "例如，如果图像中有两条狗，则实例分割需要区分像素属于的两条狗中的哪一条。\n",
    "\n",
    "### 转置卷积\n",
    "\n",
    "（1）下采样和上采用\n",
    "\n",
    "下采样：即缩小图像，常用的方法主要有：卷积、池化。第六章我们学习的卷积层和汇聚（池化）层,通常会减少输入图像的空间维度（高和宽)。\n",
    "\n",
    "上采样：即放大图像，常用的方法主要有：转置卷积、上池化（Unpooling） 。\n",
    "\n",
    "在语义分割中，我们需要对输入图像的每一像素分类。\n",
    "但下采样会使得输出图像和输入图像的空间维度不一致，所以我们在使用完下采样后还得再使用上采样使得输出图像和输入图像的空间维度相同，进而通过输出像素的类别直接判定输入像素的类别。\n",
    "\n",
    "（2）基本操作\n",
    "\n",
    "暂时忽略通道，从基本的转置卷积开始：假设我们有一个nh×nw的输入张量和一个kh×kw的卷积核。\n",
    "没有填充、以步幅为1滑动卷积核窗口，每行nw次，每列nh次，共产生nhnw个中间结果。\n",
    "每个中间结果都是一个（nh+kh-1）×（nw+kw-1）的张量。\n",
    "每个输入元素和卷积核相乘，所得结果根据输入元素的位置*指定步幅滑动。\n",
    "\n",
    "（3）填充、步幅和多通道\n",
    "\n",
    "在转置卷积中，填充被应用于输出，将减少输出的高和宽（常规卷积的填充应用于输入，将增加输入的高和宽）。\n",
    "例如，当转置卷积将高和宽两侧的填充数指定为1时，将删除输出的第一和最后的行与列。\n",
    "在转置卷积中，步幅被应用于输出，而不是输入。\n",
    "每个输入元素和卷积核相乘，所得结果根据输入元素的位置×指定步幅滑动。\n",
    "\n",
    "（4）为什么叫转置卷积?\n",
    "\n",
    "抽象来看，给定输入向量x和权重矩阵W，常规卷积的前向传播函数可以通过将其输入与权重矩阵相乘并输出向量y=Wx来实现，其反向传播函数可以通过将其输入与权重矩阵的转置WT相乘来实现。\n",
    "\n",
    "转置卷积则和常规卷积相反，它的正向传播和反向传播函数将输入向量分别与W和W相乘。\n",
    "\n",
    "### 全卷积网络\n",
    "\n",
    "语义分割是对图像中的每个像素分类。全卷积网络采用卷积神经网络实现了从图像像素到像素类别的变换。\n",
    "与我们之前在图像分类或目标检测部分介绍的卷积神经网络不同，全卷积网络引入的转置卷积将中间层特征图的高和宽变换回输入图像的尺寸。\n",
    "因此，输出的类别预测与输入图像在像素级别上具有一一对应关系：通道维的输出即该位置对应像素的类别预测。\n",
    "\n",
    "模型结构：首先，使用在ImageNet数据集上训练好的ResNet-18模型前16层作为卷积神经网络提取图像特征。\n",
    "接下来，使用1×1卷积层将输出通道数转换为Pascal VOC2012数据集的类数（21类，即每个像素有21种索引类别）。\n",
    "最后，通过转置卷积将特征图的高度和宽度增加32倍,将其变回输入图像的高和宽。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbc8e472-f79c-474b-89ab-46038199ad08",
   "metadata": {},
   "source": [
    "## 风格迁移\n",
    "\n",
    "（1）风格迁移：是指使用卷积神经网络，自动将一个图像（风格图像）的风格应用到另一图像（内容图像）之上得到合成图像。\n",
    "\n",
    "（2）方法\n",
    "\n",
    "1. 首先，初始化合成图像为内容图像。\n",
    "\n",
    "2. 然后，选择一个预训练好的卷积神经网络net（其网络参数在训练中无需更新），抽取内容图像和风格图像在net的某些层输出作为内容特征或风格特征标签。\n",
    "\n",
    "3. 在此基础上，通过前向传播（实线箭头方向）抽取合成图像的风格层、内容层和输出层输出，计算三者损失，并通过反向传播（虚线箭头方向）更新合成图像（这个才是我们训练更新的目标）。\n",
    "\n",
    "（3）风格迁移常用的损失函数由3部分组成：\n",
    "\n",
    "1. 内容损失使合成图像与内容图像在内容特征上接近。\n",
    "\n",
    "2. 风格损失使合成图像与风格图像在风格特征上接近。\n",
    "\n",
    "3. 全变分损失则有助于减少合成图像中的噪点。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fecf60e9-90cf-4cca-88c6-652cf5eff0fb",
   "metadata": {},
   "source": [
    "## 中英文翻译系统\n",
    "\n",
    "机器翻译（machine translation）指的是将序列从一种语言自动翻译成另一种语言，属于序列到序列学习问题。\n",
    "目前有两类机器翻译方法：①统计机器翻译：统计学方法②神经机器翻译：神经网络的方法。\n",
    "\n",
    "本中英文翻译系统基于序列到序列学习（seq2seq）的循环神经网络，下面是具体设计：\n",
    "\n",
    "（1）预处理数据集\n",
    "\n",
    "数据集中的每一行都是制表符分隔的文本序列对，序列对由英文文本序列和翻译后的法语文本序列组成（中文是源语言，英语是目标语言）。\n",
    "每个文本序列可以是一个句子，也可以是包含多个句子的一个段落。\n",
    "用空格代替不间断空格，使用小写字母替换大写字母，并在单词和标点符号之间插入空格。\n",
    "\n",
    "（2）词元化\n",
    "\n",
    "单词级词元化：利用空格切分。其中每个词元要么是一个词，要么是一个标点符号。\n",
    "\n",
    "（3）词表化\n",
    "\n",
    "分别为源语言和目标语言构建两个词表。为了防止词表过大，将出现次数少于2次的低频率词元视为相同的未知“”词元。\n",
    "此外，还指定了额外的特定词元，例如在小批量时用于将序列填充到相同长度的填充词元“”,以及序列开始词元“”和结束词元“”。\n",
    "\n",
    "（4）转化成适于循环神经网络处理的数据集\n",
    "\n",
    "1. 采用截断和填充定义序列长度统一化函数。\n",
    "   注意：此处的长度统一化是指将源语言序列和目标语言序列长度分别统一化，并不是说要求两类序列长度也要相同。\n",
    "\n",
    "2. 定义将序列转换成张量函数。\n",
    "   将特定的“”词元添加到所有序列的末尾，用于表示序列的结束。\n",
    "   还记录了每个文本序列的长度，统计长度时排除了填充词元。\n",
    "\n",
    "（5）训练\n",
    "\n",
    "强制教学：用特定的序列开始词元（“”）和原始的输出序列（不包括序列结束词元“”）拼接在一起作为解码器的输入。\n",
    "\n",
    "（6）预测\n",
    "\n",
    "每个解码器当前时间步的输入来自于前一时间步的预测词元。与训练类似，序列开始词元（“”）在初始时间步被输入到解码器中。\n",
    "\n",
    "（7）预测序列的评估\n",
    "\n",
    "BLEU（bilingual evaluation understudy）：通过测量预测序列和标签序列之间的n元语法的匹配度来评估预测性能。\n",
    "\n",
    "$exp(min(0,1-\\frac{len_{label}}{len_{pred}})) \\prod_{n=1}^k p_n^{1/2^n}$\n",
    "\n",
    "$len_{label}$：标签序列长度（单词数）\n",
    "\n",
    "$len_{pred}$：预测序列长度（单词数）\n",
    "\n",
    "$p_n = \\frac{\\text{预测序列的n元词词元在标签序列中数量}}{\\text{预测序列的n元词词元数}}$\n",
    "\n",
    "k是用于匹配的最长的n元语法\n",
    "\n",
    "当预测序列与标签序列完全相同时，BLEU为1，即BLEU值越大预测性能越好。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4cb64fdf-0a00-48d3-925a-842363d02ca2",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "@deathbeds/ipydrawio": {
   "xml": "<mxfile host=\"localhost\" modified=\"2024-07-07T09:05:30.546Z\" agent=\"5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36 UOS Home\" etag=\"e7rsN7x7Mqnf5I2ikoa7\" version=\"20.5.3\" type=\"embed\" pages=\"2\"><diagram id=\"ppDGj5rbBgHAk-XOpS28\" name=\"Page-1\"><mxGraphModel dx=\"1417\" dy=\"703\" grid=\"1\" gridSize=\"10\" guides=\"1\" tooltips=\"1\" connect=\"1\" arrows=\"1\" fold=\"1\" page=\"1\" pageScale=\"1\" pageWidth=\"827\" pageHeight=\"500\" math=\"0\" shadow=\"0\"><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/><mxCell id=\"2\" value=\"o1\" style=\"ellipse;aspect=fixed;fontSize=30;fillColor=#dae8fc;strokeColor=#6c8ebf;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"430\" y=\"120\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"3\" value=\"o1\" style=\"ellipse;aspect=fixed;fontSize=30;fillColor=#ffe6cc;strokeColor=#d79b00;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"270\" y=\"280\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"4\" value=\"o1\" style=\"ellipse;aspect=fixed;fontSize=30;fillColor=#ffe6cc;strokeColor=#d79b00;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"430\" y=\"280\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"5\" value=\"o1\" style=\"ellipse;aspect=fixed;fontSize=30;fillColor=#ffe6cc;strokeColor=#d79b00;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"590\" y=\"280\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"6\" value=\"\" style=\"endArrow=classic;rounded=0;fontSize=30;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0;entryY=1;entryDx=0;entryDy=0;\" parent=\"1\" source=\"3\" target=\"2\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"500\" y=\"430\" as=\"sourcePoint\"/><mxPoint x=\"550\" y=\"380\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"7\" value=\"\" style=\"endArrow=classic;rounded=0;fontSize=30;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;\" parent=\"1\" source=\"4\" target=\"2\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"320\" y=\"290\" as=\"sourcePoint\"/><mxPoint x=\"470\" y=\"210\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"8\" value=\"\" style=\"endArrow=classic;rounded=0;fontSize=30;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=1;entryY=1;entryDx=0;entryDy=0;\" parent=\"1\" source=\"5\" target=\"2\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"480\" y=\"290\" as=\"sourcePoint\"/><mxPoint x=\"480\" y=\"210\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"9\" value=\"输入层\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"140\" y=\"305\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell><mxCell id=\"10\" value=\"输出层\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"140\" y=\"145\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell></root></mxGraphModel></diagram><diagram id=\"RTb6tD45HHHSticV-QxI\" name=\"第 2 页\"><mxGraphModel dx=\"1417\" dy=\"703\" grid=\"1\" gridSize=\"10\" guides=\"1\" tooltips=\"1\" connect=\"1\" arrows=\"1\" fold=\"1\" page=\"1\" pageScale=\"1\" pageWidth=\"827\" pageHeight=\"500\" math=\"0\" shadow=\"0\"><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-1\" value=\"o1\" style=\"ellipse;aspect=fixed;fillColor=#dae8fc;strokeColor=#6c8ebf;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"360\" y=\"40\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-2\" value=\"o2\" style=\"ellipse;aspect=fixed;fillColor=#dae8fc;strokeColor=#6c8ebf;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"520\" y=\"40\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-3\" value=\"o3\" style=\"ellipse;aspect=fixed;fillColor=#dae8fc;strokeColor=#6c8ebf;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"680\" y=\"40\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-4\" value=\"h1\" style=\"ellipse;aspect=fixed;fillColor=#fff2cc;strokeColor=#d6b656;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"200\" y=\"200\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-5\" value=\"h2\" style=\"ellipse;aspect=fixed;fillColor=#fff2cc;strokeColor=#d6b656;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"360\" y=\"200\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-6\" value=\"h3\" style=\"ellipse;aspect=fixed;fillColor=#fff2cc;strokeColor=#d6b656;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"520\" y=\"200\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-7\" value=\"h4\" style=\"ellipse;aspect=fixed;fillColor=#fff2cc;strokeColor=#d6b656;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"680\" y=\"200\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-8\" value=\"h5\" style=\"ellipse;aspect=fixed;fillColor=#fff2cc;strokeColor=#d6b656;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"840\" y=\"200\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-9\" value=\"x1\" style=\"ellipse;aspect=fixed;fillColor=#ffe6cc;strokeColor=#d79b00;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"280\" y=\"360\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-10\" value=\"x2\" style=\"ellipse;aspect=fixed;fillColor=#ffe6cc;strokeColor=#d79b00;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"440\" y=\"360\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-11\" value=\"x3\" style=\"ellipse;aspect=fixed;fillColor=#ffe6cc;strokeColor=#d79b00;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"600\" y=\"360\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-12\" value=\"x4\" style=\"ellipse;aspect=fixed;fillColor=#ffe6cc;strokeColor=#d79b00;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"760\" y=\"360\" width=\"80\" height=\"80\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-13\" value=\"\" style=\"endArrow=classic;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;\" parent=\"1\" source=\"Qcui9y9y7JmnU8C_Asa0-9\" target=\"Qcui9y9y7JmnU8C_Asa0-4\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"560\" y=\"270\" as=\"sourcePoint\"/><mxPoint x=\"610\" y=\"220\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-14\" value=\"\" style=\"endArrow=classic;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;\" parent=\"1\" source=\"Qcui9y9y7JmnU8C_Asa0-9\" target=\"Qcui9y9y7JmnU8C_Asa0-5\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"330\" y=\"370\" as=\"sourcePoint\"/><mxPoint x=\"250\" y=\"290\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-15\" value=\"\" style=\"endArrow=classic;rounded=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;\" parent=\"1\" source=\"Qcui9y9y7JmnU8C_Asa0-9\" target=\"Qcui9y9y7JmnU8C_Asa0-6\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"400\" y=\"360\" as=\"sourcePoint\"/><mxPoint x=\"410\" y=\"290\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-16\" value=\"\" style=\"endArrow=classic;rounded=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;\" parent=\"1\" source=\"Qcui9y9y7JmnU8C_Asa0-9\" target=\"Qcui9y9y7JmnU8C_Asa0-7\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"330\" y=\"370\" as=\"sourcePoint\"/><mxPoint x=\"570\" y=\"290\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-17\" value=\"\" style=\"endArrow=classic;rounded=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;\" parent=\"1\" source=\"Qcui9y9y7JmnU8C_Asa0-9\" target=\"Qcui9y9y7JmnU8C_Asa0-8\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"390\" y=\"380\" as=\"sourcePoint\"/><mxPoint x=\"730\" y=\"290\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-18\" value=\"\" style=\"endArrow=classic;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;\" parent=\"1\" source=\"Qcui9y9y7JmnU8C_Asa0-4\" target=\"Qcui9y9y7JmnU8C_Asa0-1\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"330\" y=\"370\" as=\"sourcePoint\"/><mxPoint x=\"250\" y=\"290\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-19\" value=\"\" style=\"endArrow=classic;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;\" parent=\"1\" source=\"Qcui9y9y7JmnU8C_Asa0-4\" target=\"Qcui9y9y7JmnU8C_Asa0-2\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"250\" y=\"210\" as=\"sourcePoint\"/><mxPoint x=\"410\" y=\"130\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-20\" value=\"\" style=\"endArrow=classic;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;\" parent=\"1\" source=\"Qcui9y9y7JmnU8C_Asa0-4\" target=\"Qcui9y9y7JmnU8C_Asa0-3\" edge=\"1\"><mxGeometry width=\"50\" height=\"50\" relative=\"1\" as=\"geometry\"><mxPoint x=\"250\" y=\"210\" as=\"sourcePoint\"/><mxPoint x=\"570\" y=\"130\" as=\"targetPoint\"/></mxGeometry></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-21\" value=\"...\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"390\" y=\"170\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-22\" value=\"...\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"550\" y=\"170\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-23\" value=\"...\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"710\" y=\"170\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-24\" value=\"...\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"870\" y=\"170\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-25\" value=\"...\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"470\" y=\"330\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-26\" value=\"...\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"630\" y=\"330\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-27\" value=\"...\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"790\" y=\"330\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-30\" value=\"输出层\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"80\" y=\"60\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-31\" value=\"隐藏层\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"80\" y=\"220\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell><mxCell id=\"Qcui9y9y7JmnU8C_Asa0-32\" value=\"输入层\" style=\"text;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;fontSize=30;\" parent=\"1\" vertex=\"1\"><mxGeometry x=\"80\" y=\"380\" width=\"60\" height=\"30\" as=\"geometry\"/></mxCell></root></mxGraphModel></diagram></mxfile>"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
